Jul 15 19:54:12.200586 ip-10-0-132-12 systemd[1]: Starting Kubernetes Kubelet... Jul 15 19:54:12.601006 ip-10-0-132-12 kubenswrapper[2557]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:54:12.601006 ip-10-0-132-12 kubenswrapper[2557]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 19:54:12.601006 ip-10-0-132-12 kubenswrapper[2557]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:54:12.601006 ip-10-0-132-12 kubenswrapper[2557]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 19:54:12.601006 ip-10-0-132-12 kubenswrapper[2557]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 19:54:12.601930 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.601843 2557 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 19:54:12.606918 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606902 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:54:12.606918 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606919 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606922 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606925 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606928 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606931 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606934 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606936 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606939 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606942 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606944 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606947 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606950 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606952 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606956 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606961 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606965 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606969 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606972 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606975 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:54:12.606981 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606988 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606991 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606994 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.606996 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607005 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607008 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607019 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607023 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607025 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607028 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607030 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607033 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607035 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607038 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607041 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607043 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607046 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607048 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607051 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607053 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:54:12.607451 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607056 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607058 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607060 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607063 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607065 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607068 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607070 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607073 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607075 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607078 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607080 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607082 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607085 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607094 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607098 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607101 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607104 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607106 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607109 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607111 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607114 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:54:12.607940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607117 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607119 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607122 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607125 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607129 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607131 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607134 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607137 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607140 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607142 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607145 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607147 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607149 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607152 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607154 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607157 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607159 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607162 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607164 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:54:12.608458 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607167 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607170 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607173 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607175 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607177 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607181 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607184 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607187 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607190 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607193 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607196 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607199 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607201 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607204 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607206 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607209 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607212 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607214 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.607216 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608048 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:54:12.608950 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608055 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608058 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608061 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608064 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608068 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608072 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608075 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608078 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608081 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608084 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608087 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608090 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608092 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608095 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608098 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608100 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608103 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608106 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608109 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608123 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:54:12.609438 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608126 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608129 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608131 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608134 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608137 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608139 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608142 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608145 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608149 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608151 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608154 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608157 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608159 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608162 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608164 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608167 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608170 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608173 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608176 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:54:12.609921 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608178 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608181 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608184 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608186 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608189 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608191 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608194 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608196 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608198 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608201 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608204 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608208 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608211 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608219 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608221 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608224 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608226 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608230 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608248 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608251 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:54:12.610395 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608254 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608256 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608259 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608261 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608264 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608267 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608269 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608272 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608274 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608277 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608279 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608282 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608284 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608286 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608289 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608291 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608294 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608296 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608299 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:54:12.610883 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608301 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608304 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608306 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608310 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608313 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608316 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608318 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608321 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608324 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608327 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608329 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608333 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608336 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608339 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608342 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608344 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608346 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608349 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608351 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:54:12.611404 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608354 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608768 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608774 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608777 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608782 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608786 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608789 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608791 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608795 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608799 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608802 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608805 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608808 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608811 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608813 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608816 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608819 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608822 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608825 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:54:12.611865 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608828 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608836 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608839 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608842 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608844 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608847 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608849 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608852 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608854 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608857 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608859 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608862 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608864 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608867 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608869 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608871 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608874 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608876 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608879 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:54:12.612328 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608881 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608884 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608886 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608889 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608891 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608896 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608899 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608902 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608905 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608908 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608911 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608914 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608917 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608920 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608922 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608930 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608932 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608935 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608938 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:54:12.612779 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608940 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608944 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608946 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608949 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608951 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608954 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608957 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608959 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608962 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608964 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608967 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608969 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608972 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608974 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608977 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608980 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608982 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608985 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608987 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608990 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:54:12.613243 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608992 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608995 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.608998 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609000 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609003 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609005 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609008 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609010 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609013 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609016 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609018 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609021 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609023 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609026 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609028 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609031 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609036 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609038 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609040 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609043 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:54:12.613766 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609046 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609051 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609054 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609119 2557 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609130 2557 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609137 2557 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609141 2557 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609145 2557 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609148 2557 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609152 2557 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609156 2557 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609159 2557 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609163 2557 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609166 2557 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609169 2557 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609172 2557 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609175 2557 flags.go:64] FLAG: --cgroup-root="" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609178 2557 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609181 2557 flags.go:64] FLAG: --client-ca-file="" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609184 2557 flags.go:64] FLAG: --cloud-provider="external" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609187 2557 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609191 2557 flags.go:64] FLAG: --cluster-domain="" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609194 2557 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609197 2557 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 19:54:12.614370 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609200 2557 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609203 2557 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609207 2557 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609210 2557 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609214 2557 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609218 2557 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609221 2557 flags.go:64] FLAG: --contention-profiling="false" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609223 2557 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609226 2557 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609229 2557 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609247 2557 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609253 2557 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609258 2557 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609263 2557 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609266 2557 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609269 2557 flags.go:64] FLAG: --enable-server="true" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609272 2557 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609277 2557 flags.go:64] FLAG: --event-burst="100" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609280 2557 flags.go:64] FLAG: --event-qps="50" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609283 2557 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609286 2557 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609288 2557 flags.go:64] FLAG: --eviction-hard="" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609292 2557 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609295 2557 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609299 2557 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 19:54:12.614940 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609302 2557 flags.go:64] FLAG: --eviction-soft="" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609305 2557 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609308 2557 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609311 2557 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609314 2557 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609317 2557 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609320 2557 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609323 2557 flags.go:64] FLAG: --feature-gates="" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609327 2557 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609330 2557 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609334 2557 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609341 2557 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609344 2557 flags.go:64] FLAG: --healthz-port="10248" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609347 2557 flags.go:64] FLAG: --help="false" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609350 2557 flags.go:64] FLAG: --hostname-override="ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609353 2557 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609356 2557 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609359 2557 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609362 2557 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609365 2557 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609368 2557 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609371 2557 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609374 2557 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 19:54:12.615589 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609377 2557 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609379 2557 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609383 2557 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609385 2557 flags.go:64] FLAG: --kube-reserved="" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609388 2557 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609391 2557 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609394 2557 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609396 2557 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609399 2557 flags.go:64] FLAG: --lock-file="" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609402 2557 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609405 2557 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609408 2557 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609413 2557 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609416 2557 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609419 2557 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609421 2557 flags.go:64] FLAG: --logging-format="text" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609427 2557 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609430 2557 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609433 2557 flags.go:64] FLAG: --manifest-url="" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609435 2557 flags.go:64] FLAG: --manifest-url-header="" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609442 2557 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609445 2557 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609449 2557 flags.go:64] FLAG: --max-pods="110" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609452 2557 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609455 2557 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 19:54:12.616169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609458 2557 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609461 2557 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609464 2557 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609466 2557 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609469 2557 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609476 2557 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609479 2557 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609482 2557 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609485 2557 flags.go:64] FLAG: --pod-cidr="" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609488 2557 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609492 2557 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609495 2557 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609498 2557 flags.go:64] FLAG: --pods-per-core="0" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609501 2557 flags.go:64] FLAG: --port="10250" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609504 2557 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609507 2557 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06cc4794b3030749d" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609511 2557 flags.go:64] FLAG: --qos-reserved="" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609514 2557 flags.go:64] FLAG: --read-only-port="10255" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609516 2557 flags.go:64] FLAG: --register-node="true" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609519 2557 flags.go:64] FLAG: --register-with-taints="" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609523 2557 flags.go:64] FLAG: --registry-burst="10" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609526 2557 flags.go:64] FLAG: --registry-qps="5" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609528 2557 flags.go:64] FLAG: --reserved-cpus="" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609531 2557 flags.go:64] FLAG: --reserved-memory="" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609536 2557 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 19:54:12.617074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609539 2557 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609542 2557 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609545 2557 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609548 2557 flags.go:64] FLAG: --runonce="false" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609551 2557 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609553 2557 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609556 2557 flags.go:64] FLAG: --seccomp-default="false" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609560 2557 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609562 2557 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609565 2557 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609568 2557 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609571 2557 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609574 2557 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609577 2557 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609580 2557 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609583 2557 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609585 2557 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609588 2557 flags.go:64] FLAG: --system-cgroups="" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609591 2557 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609596 2557 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609599 2557 flags.go:64] FLAG: --tls-cert-file="" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609601 2557 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609605 2557 flags.go:64] FLAG: --tls-min-version="" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609609 2557 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609612 2557 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 19:54:12.617826 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609615 2557 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609617 2557 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609620 2557 flags.go:64] FLAG: --v="2" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609624 2557 flags.go:64] FLAG: --version="false" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609628 2557 flags.go:64] FLAG: --vmodule="" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609634 2557 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.609637 2557 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609752 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609755 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609758 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609761 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609764 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609767 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609769 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609772 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609774 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609787 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609791 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609827 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609832 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609837 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:54:12.618443 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609842 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609845 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609848 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609851 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609854 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609857 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609860 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609863 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609865 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609868 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609871 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609874 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609876 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609879 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609881 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609884 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609888 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609891 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609894 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609897 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:54:12.618951 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609899 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609902 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609905 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609908 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609910 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609913 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609915 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609918 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609921 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609924 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609928 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609930 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609933 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609936 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609938 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609941 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609943 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609946 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609949 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:54:12.619476 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609952 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609954 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609957 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609959 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609962 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609964 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609967 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609969 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609972 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609975 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609978 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609981 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609983 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609986 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609988 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609991 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609994 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609996 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.609999 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610002 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:54:12.619940 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610004 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610007 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610010 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610013 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610015 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610018 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610021 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610026 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610029 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610032 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610036 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610039 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610041 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610045 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610047 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610052 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610055 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610058 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610060 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:54:12.620436 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610063 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610066 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610069 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610072 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610075 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610077 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.610080 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.610086 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.618333 2557 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.618350 2557 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.618370 2557 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.618376 2557 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618433 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618439 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618442 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:54:12.620890 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618445 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618448 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618451 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618454 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618458 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618460 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618463 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618465 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618468 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618472 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618476 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618479 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618482 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618485 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618487 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618490 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618493 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618495 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618499 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618501 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:54:12.621340 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618504 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618506 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618509 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618511 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618514 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618516 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618520 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618523 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618526 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618528 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618531 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618533 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618536 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618539 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618541 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618544 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618547 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618549 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618552 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618554 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:54:12.621825 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618556 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618559 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618561 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618564 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618566 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618569 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618571 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618574 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618576 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618579 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618583 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618586 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618589 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618592 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618594 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618597 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618599 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618602 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618605 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:54:12.622329 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618609 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618612 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618615 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618618 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618620 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618623 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618625 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618628 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618630 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618633 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618635 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618638 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618640 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618643 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618645 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618647 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618650 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618653 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618656 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:54:12.622790 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618659 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618661 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618663 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618666 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618669 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618671 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618674 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618677 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618679 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618682 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618684 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618687 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618690 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618693 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618695 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618698 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618700 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:54:12.623322 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618703 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.618708 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618817 2557 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618822 2557 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618825 2557 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618829 2557 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618833 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618837 2557 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618840 2557 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618843 2557 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618846 2557 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618849 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618852 2557 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618855 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 19:54:12.623719 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618858 2557 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618861 2557 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618863 2557 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618866 2557 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618868 2557 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618871 2557 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618875 2557 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618878 2557 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618881 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618884 2557 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618887 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618890 2557 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618893 2557 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618896 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618899 2557 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618901 2557 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618904 2557 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618907 2557 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618909 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 19:54:12.624078 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618912 2557 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618915 2557 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618918 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618920 2557 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618922 2557 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618925 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618927 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618930 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618932 2557 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618934 2557 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618937 2557 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618941 2557 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618944 2557 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618947 2557 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618950 2557 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618953 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618955 2557 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618958 2557 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618960 2557 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618963 2557 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 19:54:12.624553 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618965 2557 feature_gate.go:328] unrecognized feature gate: Example Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618968 2557 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618970 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618973 2557 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618975 2557 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618978 2557 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618980 2557 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618984 2557 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618987 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618990 2557 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618992 2557 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618994 2557 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.618997 2557 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619000 2557 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619002 2557 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619005 2557 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619007 2557 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619018 2557 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619022 2557 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619024 2557 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 19:54:12.625051 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619027 2557 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619029 2557 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619032 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619035 2557 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619037 2557 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619040 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619042 2557 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619046 2557 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619049 2557 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619052 2557 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619054 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619056 2557 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619059 2557 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619061 2557 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619064 2557 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619066 2557 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619069 2557 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619072 2557 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619074 2557 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619080 2557 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 19:54:12.625552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619082 2557 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619085 2557 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619087 2557 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619090 2557 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619093 2557 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619095 2557 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619098 2557 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:12.619100 2557 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.619105 2557 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.619215 2557 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.621190 2557 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.622563 2557 server.go:1019] "Starting client certificate rotation" Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.622666 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:54:12.626056 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.622707 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 19:54:12.626501 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.624960 2557 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:54:12.628505 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.628488 2557 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 19:54:12.636744 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.636727 2557 log.go:25] "Validated CRI v1 runtime API" Jul 15 19:54:12.639620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.639599 2557 log.go:25] "Validated CRI v1 image API" Jul 15 19:54:12.640413 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.640397 2557 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 19:54:12.643045 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.643008 2557 fs.go:135] Filesystem UUIDs: map[4a5024b6-140c-456d-99e1-f665d4c16490:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 c902d479-d906-4019-9ce4-d371d31c5041:/dev/nvme0n1p3] Jul 15 19:54:12.643103 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.643046 2557 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 19:54:12.648613 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.648486 2557 manager.go:216] Machine: {Timestamp:2026-07-15 19:54:12.647163895 +0000 UTC m=+0.317950411 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099672 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e6004b8d60b7dc82f496cf829bc7a SystemUUID:ec2e6004-b8d6-0b7d-c82f-496cf829bc7a BootID:b26f62e9-05f4-48e6-b133-259375f7e34d Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:d9:bc:74:f8:79 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:d9:bc:74:f8:79 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:a0:84:ea:b0:da Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 19:54:12.648613 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.648605 2557 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 19:54:12.648721 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.648690 2557 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 19:54:12.651525 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.651496 2557 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 19:54:12.651680 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.651527 2557 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-12.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 19:54:12.652309 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.652298 2557 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 19:54:12.652345 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.652313 2557 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 19:54:12.652345 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.652329 2557 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:54:12.653046 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.653035 2557 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 19:54:12.654536 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.654525 2557 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 19:54:12.657391 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.657380 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:54:12.657529 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.657519 2557 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 19:54:12.658157 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658147 2557 kubelet.go:486] "Attempting to sync node with API server" Jul 15 19:54:12.658195 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658180 2557 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 19:54:12.658227 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658200 2557 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 19:54:12.658227 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658212 2557 kubelet.go:398] "Adding apiserver pod source" Jul 15 19:54:12.658227 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658221 2557 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 19:54:12.658638 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658626 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:54:12.658679 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.658647 2557 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 19:54:12.661612 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.661591 2557 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 19:54:12.663255 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663225 2557 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 19:54:12.663303 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663274 2557 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 19:54:12.663898 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663886 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 19:54:12.663933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663905 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 19:54:12.663933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663915 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 19:54:12.663933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663920 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 19:54:12.663933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663934 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663940 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663948 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663956 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663966 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663971 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663980 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 19:54:12.664040 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.663994 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 19:54:12.664647 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.664638 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 19:54:12.664647 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.664647 2557 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 19:54:12.668251 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.668220 2557 server.go:1268] "Started kubelet" Jul 15 19:54:12.668358 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.668325 2557 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 19:54:12.668449 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.668387 2557 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 19:54:12.668500 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.668476 2557 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 19:54:12.669255 ip-10-0-132-12 systemd[1]: Started Kubernetes Kubelet. Jul 15 19:54:12.669549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.669533 2557 server.go:310] "Adding debug handlers to kubelet server" Jul 15 19:54:12.669923 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.669756 2557 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 19:54:12.669923 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.669883 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:54:12.672218 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.672187 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 19:54:12.672347 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.672217 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 19:54:12.672347 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.672268 2557 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-12.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 19:54:12.673673 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.673655 2557 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 19:54:12.674435 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.674419 2557 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 19:54:12.674500 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.674433 2557 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 19:54:12.674549 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.674530 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:12.675069 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675053 2557 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 19:54:12.675069 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675054 2557 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 19:54:12.675201 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675081 2557 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 19:54:12.675201 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675197 2557 reconstruct.go:97] "Volume reconstruction finished" Jul 15 19:54:12.675334 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675205 2557 reconciler.go:29] "Reconciler: start to sync state" Jul 15 19:54:12.675334 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675289 2557 factory.go:55] Registering systemd factory Jul 15 19:54:12.675334 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675301 2557 factory.go:223] Registration of the systemd container factory successfully Jul 15 19:54:12.675456 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.674460 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78cfcada8 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.66819012 +0000 UTC m=+0.338976636,LastTimestamp:2026-07-15 19:54:12.66819012 +0000 UTC m=+0.338976636,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.675526 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675513 2557 factory.go:153] Registering CRI-O factory Jul 15 19:54:12.675570 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675531 2557 factory.go:223] Registration of the crio container factory successfully Jul 15 19:54:12.675613 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675581 2557 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 19:54:12.675613 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675603 2557 factory.go:103] Registering Raw factory Jul 15 19:54:12.675683 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.675620 2557 manager.go:1196] Started watching for new ooms in manager Jul 15 19:54:12.676031 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.676019 2557 manager.go:349] Starting recovery of all containers Jul 15 19:54:12.681444 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.681271 2557 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-12.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 19:54:12.681444 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.681390 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 19:54:12.688973 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.688958 2557 manager.go:354] Recovery completed Jul 15 19:54:12.690121 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.690101 2557 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory Jul 15 19:54:12.693000 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.692988 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.694967 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.694935 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.695041 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.694988 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.695041 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.695003 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.695621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.695607 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.695688 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.695607 2557 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 19:54:12.695688 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.695647 2557 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 19:54:12.695688 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.695665 2557 state_mem.go:36] "Initialized new in-memory state store" Jul 15 19:54:12.696354 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.696339 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.696423 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.696366 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.696423 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.696377 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.696911 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.696898 2557 policy_none.go:49] "None policy: Start" Jul 15 19:54:12.696939 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.696918 2557 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 19:54:12.696973 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.696936 2557 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 19:54:12.697166 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.697107 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78e954577 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-12.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.694967671 +0000 UTC m=+0.365754196,LastTimestamp:2026-07-15 19:54:12.694967671 +0000 UTC m=+0.365754196,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.697917 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.697906 2557 policy_none.go:47] "Start" Jul 15 19:54:12.708424 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.708348 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78e95b4bf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-12.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.694996159 +0000 UTC m=+0.365782675,LastTimestamp:2026-07-15 19:54:12.694996159 +0000 UTC m=+0.365782675,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.720310 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.720208 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78e95df05 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-12.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.695006981 +0000 UTC m=+0.365793498,LastTimestamp:2026-07-15 19:54:12.695006981 +0000 UTC m=+0.365793498,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.729489 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.729426 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-12.ec2.internal.18c28ea78e954577\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78e954577 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-12.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.694967671 +0000 UTC m=+0.365754196,LastTimestamp:2026-07-15 19:54:12.696352238 +0000 UTC m=+0.367138753,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.734802 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.734789 2557 manager.go:337] "Starting Device Plugin manager" Jul 15 19:54:12.734862 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.734821 2557 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 19:54:12.734862 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.734832 2557 server.go:87] "Starting device plugin registration server" Jul 15 19:54:12.735122 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.735107 2557 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 19:54:12.735196 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.735126 2557 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 19:54:12.735326 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.735198 2557 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 19:54:12.735326 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.735303 2557 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 19:54:12.735326 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.735313 2557 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 19:54:12.735825 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.735804 2557 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 19:54:12.735892 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.735855 2557 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:12.741469 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.741412 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-12.ec2.internal.18c28ea78e95b4bf\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78e95b4bf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-132-12.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.694996159 +0000 UTC m=+0.365782675,LastTimestamp:2026-07-15 19:54:12.696370562 +0000 UTC m=+0.367157078,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.751149 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.751078 2557 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-132-12.ec2.internal.18c28ea78e95df05\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea78e95df05 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-132-12.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.695006981 +0000 UTC m=+0.365793498,LastTimestamp:2026-07-15 19:54:12.696382917 +0000 UTC m=+0.367169434,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.755829 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.755811 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbwxq" Jul 15 19:54:12.761747 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.761680 2557 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-12.ec2.internal.18c28ea791183ac6 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-12.ec2.internal,UID:ip-10-0-132-12.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeAllocatableEnforced,Message:Updated Node Allocatable limit across pods,Source:EventSource{Component:kubelet,Host:ip-10-0-132-12.ec2.internal,},FirstTimestamp:2026-07-15 19:54:12.737104582 +0000 UTC m=+0.407891101,LastTimestamp:2026-07-15 19:54:12.737104582 +0000 UTC m=+0.407891101,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-12.ec2.internal,}" Jul 15 19:54:12.766298 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.766277 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cbwxq" Jul 15 19:54:12.777506 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.776866 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 19:54:12.778835 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.778817 2557 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 19:54:12.778888 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.778865 2557 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 19:54:12.778921 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.778891 2557 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 19:54:12.778951 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.778932 2557 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 19:54:12.788785 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.788764 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:54:12.835910 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.835878 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.837151 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.837134 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.837281 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.837165 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.837281 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.837176 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.837281 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.837198 2557 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.848876 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.848858 2557 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.848984 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.848884 2557 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-12.ec2.internal\": node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:12.879841 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.879769 2557 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal"] Jul 15 19:54:12.879841 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.879829 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.881558 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.881541 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.881653 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.881576 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.881653 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.881588 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.882980 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.882968 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.883096 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.883081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.883138 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.883119 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.883885 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.883867 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:12.884386 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.884372 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.884480 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.884396 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.884480 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.884406 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.884480 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.884416 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.884480 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.884440 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.884480 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.884454 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.885599 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.885583 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.885647 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.885612 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:12.887698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.887684 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:12.887750 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.887709 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:12.887750 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:12.887723 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:12.915797 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.915775 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-12.ec2.internal\" not found" node="ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.920074 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.920059 2557 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-12.ec2.internal\" not found" node="ip-10-0-132-12.ec2.internal" Jul 15 19:54:12.984293 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:12.984256 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.076952 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.076912 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a843563177c22b96d13b44f8b4e55822-config\") pod \"kube-apiserver-proxy-ip-10-0-132-12.ec2.internal\" (UID: \"a843563177c22b96d13b44f8b4e55822\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.076952 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.076949 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dfe48926f4c41d3cac7cad190b9f419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal\" (UID: \"5dfe48926f4c41d3cac7cad190b9f419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.077154 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.076979 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dfe48926f4c41d3cac7cad190b9f419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal\" (UID: \"5dfe48926f4c41d3cac7cad190b9f419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.084986 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.084958 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.177758 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.177689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a843563177c22b96d13b44f8b4e55822-config\") pod \"kube-apiserver-proxy-ip-10-0-132-12.ec2.internal\" (UID: \"a843563177c22b96d13b44f8b4e55822\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.177758 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.177725 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dfe48926f4c41d3cac7cad190b9f419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal\" (UID: \"5dfe48926f4c41d3cac7cad190b9f419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.177758 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.177741 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dfe48926f4c41d3cac7cad190b9f419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal\" (UID: \"5dfe48926f4c41d3cac7cad190b9f419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.177924 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.177796 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5dfe48926f4c41d3cac7cad190b9f419-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal\" (UID: \"5dfe48926f4c41d3cac7cad190b9f419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.177924 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.177803 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a843563177c22b96d13b44f8b4e55822-config\") pod \"kube-apiserver-proxy-ip-10-0-132-12.ec2.internal\" (UID: \"a843563177c22b96d13b44f8b4e55822\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.177924 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.177812 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5dfe48926f4c41d3cac7cad190b9f419-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal\" (UID: \"5dfe48926f4c41d3cac7cad190b9f419\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.185791 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.185765 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.218989 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.218961 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.223444 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.223222 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:13.286196 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.286163 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.386713 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.386680 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.487322 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.487256 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.587874 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.587846 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.622032 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.621999 2557 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 19:54:13.622763 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.622167 2557 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 19:54:13.683112 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:13.683081 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dfe48926f4c41d3cac7cad190b9f419.slice/crio-d0d359e18e24a79c99f84070c3254c3fdadfc8300ff718f689ace16b3f1f714d WatchSource:0}: Error finding container d0d359e18e24a79c99f84070c3254c3fdadfc8300ff718f689ace16b3f1f714d: Status 404 returned error can't find the container with id d0d359e18e24a79c99f84070c3254c3fdadfc8300ff718f689ace16b3f1f714d Jul 15 19:54:13.686722 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.686707 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 19:54:13.688287 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.688271 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.770825 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.770719 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 19:49:12 +0000 UTC" deadline="2028-03-11 23:27:19.516200013 +0000 UTC" Jul 15 19:54:13.770825 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.770760 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14523h33m5.745443272s" Jul 15 19:54:13.782354 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.782310 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" event={"ID":"5dfe48926f4c41d3cac7cad190b9f419","Type":"ContainerStarted","Data":"d0d359e18e24a79c99f84070c3254c3fdadfc8300ff718f689ace16b3f1f714d"} Jul 15 19:54:13.788390 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.788372 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.860575 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.860556 2557 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 19:54:13.861091 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:13.861065 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda843563177c22b96d13b44f8b4e55822.slice/crio-16a00c1d74cfa153ebaaee908789f7294f0e4bcee0f3667dc235d8bb2e3daf89 WatchSource:0}: Error finding container 16a00c1d74cfa153ebaaee908789f7294f0e4bcee0f3667dc235d8bb2e3daf89: Status 404 returned error can't find the container with id 16a00c1d74cfa153ebaaee908789f7294f0e4bcee0f3667dc235d8bb2e3daf89 Jul 15 19:54:13.861621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.861582 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientMemory" Jul 15 19:54:13.861621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.861610 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 19:54:13.861701 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:13.861623 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeHasSufficientPID" Jul 15 19:54:13.889149 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.889129 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:13.989913 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:13.989878 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:14.050938 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.050848 2557 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:54:14.090964 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.090932 2557 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-12.ec2.internal\" not found" Jul 15 19:54:14.129472 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.129443 2557 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:54:14.159753 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.159725 2557 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:54:14.174971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.174943 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" Jul 15 19:54:14.194114 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.194084 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:54:14.194959 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.194941 2557 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" Jul 15 19:54:14.205861 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.205833 2557 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 19:54:14.660766 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.660728 2557 apiserver.go:52] "Watching apiserver" Jul 15 19:54:14.671159 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.671131 2557 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 19:54:14.672339 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.672304 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-gs6nt","kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal","openshift-dns/node-resolver-4gk5v","openshift-image-registry/node-ca-5l4v9","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal","openshift-multus/multus-4vv77","openshift-multus/multus-additional-cni-plugins-ppbdn","openshift-network-operator/iptables-alerter-wltjx","kube-system/global-pull-secret-syncer-tr84l","kube-system/konnectivity-agent-72vbr","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46","openshift-cluster-node-tuning-operator/tuned-kj66r","openshift-multus/network-metrics-daemon-ftj7m","openshift-network-diagnostics/network-check-target-45bw2"] Jul 15 19:54:14.673907 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.673886 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.674605 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.674587 2557 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 19:54:14.675200 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.675169 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.676242 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.676223 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.677564 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.677545 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.678729 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.678712 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:14.678830 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.678795 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:14.679960 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.679939 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ldplg\"" Jul 15 19:54:14.679960 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.679944 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 19:54:14.680107 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.680076 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.684359 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.681410 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.684359 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.681517 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:14.684791 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.684770 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.685413 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.685392 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.685592 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.685571 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 19:54:14.685592 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.685586 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.686193 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686174 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-registration-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686292 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686206 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-socket-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686292 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686232 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686292 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c8296fea-0715-4071-8570-f4301f99baf7-agent-certs\") pod \"konnectivity-agent-72vbr\" (UID: \"c8296fea-0715-4071-8570-f4301f99baf7\") " pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.686459 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c8296fea-0715-4071-8570-f4301f99baf7-konnectivity-ca\") pod \"konnectivity-agent-72vbr\" (UID: \"c8296fea-0715-4071-8570-f4301f99baf7\") " pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.686459 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686342 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686459 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686377 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-sys-fs\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686459 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686407 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcbb4\" (UniqueName: \"kubernetes.io/projected/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-kube-api-access-jcbb4\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686459 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686436 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad9aa6c3-01ba-48c8-a500-1468298afb49-host-slash\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.686701 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7dvp\" (UniqueName: \"kubernetes.io/projected/ad9aa6c3-01ba-48c8-a500-1468298afb49-kube-api-access-v7dvp\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.686701 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686485 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-device-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.686701 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686510 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad9aa6c3-01ba-48c8-a500-1468298afb49-iptables-alerter-script\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.686701 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.686618 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.688044 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.688027 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.689504 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.689459 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:14.689606 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.689554 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.689606 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.689573 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:14.691217 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.691201 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 19:54:14.691328 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.691226 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.691605 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.691588 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 19:54:14.703666 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.703645 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 19:54:14.703773 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.703665 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.703977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.703957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 19:54:14.704063 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.703957 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.704305 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704276 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.704393 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704315 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 19:54:14.704393 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704353 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 19:54:14.704495 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704429 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9ptfk\"" Jul 15 19:54:14.704495 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704312 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 19:54:14.704586 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704549 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 19:54:14.704648 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704607 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 19:54:14.704648 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704618 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.704749 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704734 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-89xvb\"" Jul 15 19:54:14.704749 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704746 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-zhzz2\"" Jul 15 19:54:14.704841 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704767 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.704841 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704775 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.704841 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704746 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 19:54:14.704983 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704853 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.704983 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.704888 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.705639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.705622 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-8772t\"" Jul 15 19:54:14.717429 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717405 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-xw9vs\"" Jul 15 19:54:14.717539 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717475 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.717539 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717484 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 19:54:14.717539 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717526 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2fkml\"" Jul 15 19:54:14.717699 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717533 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 19:54:14.717699 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717405 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 19:54:14.717811 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717744 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nxm5c\"" Jul 15 19:54:14.717862 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.717835 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-g642n\"" Jul 15 19:54:14.742862 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.742841 2557 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 19:54:14.754644 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.754618 2557 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-hhmgb" Jul 15 19:54:14.776638 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.776613 2557 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 19:54:14.778760 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.778740 2557 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-hhmgb" Jul 15 19:54:14.786808 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786786 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-log-socket\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.786913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786813 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-cni-bin\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.786913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786830 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-system-cni-dir\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.786913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786847 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-cnibin\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.786913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786863 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-dbus\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.786913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786879 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-var-lib-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.786913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786894 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-var-lib-kubelet\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786926 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/197197c7-6df9-4853-87c8-d67484c47710-tmp\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786941 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786956 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-kubelet\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.786993 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-systemd-units\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787033 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-system-cni-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z8mzm\" (UniqueName: \"kubernetes.io/projected/38712c2b-cfb9-4780-9720-5dcdbd748470-kube-api-access-z8mzm\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787107 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-kubelet-config\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787151 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc31ca06-d978-4248-98fd-000e3fc287b0-ovn-node-metrics-cert\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787188 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c14a3660-2a88-490e-b481-6a414a642760-serviceca\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787215 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-os-release\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787317 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-cnibin\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c8296fea-0715-4071-8570-f4301f99baf7-konnectivity-ca\") pod \"konnectivity-agent-72vbr\" (UID: \"c8296fea-0715-4071-8570-f4301f99baf7\") " pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787391 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7dvp\" (UniqueName: \"kubernetes.io/projected/ad9aa6c3-01ba-48c8-a500-1468298afb49-kube-api-access-v7dvp\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787410 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-cni-netd\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787434 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-kubernetes\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787535 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-systemd\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.787571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787563 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-cni-bin\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787609 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787627 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttsh2\" (UniqueName: \"kubernetes.io/projected/fc31ca06-d978-4248-98fd-000e3fc287b0-kube-api-access-ttsh2\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787646 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ws6qj\" (UniqueName: \"kubernetes.io/projected/197197c7-6df9-4853-87c8-d67484c47710-kube-api-access-ws6qj\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787661 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-slash\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787701 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-systemd\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787738 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787790 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-run\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787818 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0303819b-d5e8-4964-94ed-2b90452a5036-tmp-dir\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787842 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:14.787871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787866 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-cni-binary-copy\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787890 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-ovnkube-config\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787922 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysctl-conf\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787942 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtn9s\" (UniqueName: \"kubernetes.io/projected/a9563bf9-bcc6-42c8-8a50-e54da8128432-kube-api-access-gtn9s\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787964 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c8296fea-0715-4071-8570-f4301f99baf7-konnectivity-ca\") pod \"konnectivity-agent-72vbr\" (UID: \"c8296fea-0715-4071-8570-f4301f99baf7\") " pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787983 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" event={"ID":"a843563177c22b96d13b44f8b4e55822","Type":"ContainerStarted","Data":"16a00c1d74cfa153ebaaee908789f7294f0e4bcee0f3667dc235d8bb2e3daf89"} Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.787971 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-sys-fs\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788033 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-ovnkube-script-lib\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788035 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-sys-fs\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788064 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-device-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788112 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-node-log\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788127 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788172 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-device-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788200 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c14a3660-2a88-490e-b481-6a414a642760-host\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788219 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glzvz\" (UniqueName: \"kubernetes.io/projected/0303819b-d5e8-4964-94ed-2b90452a5036-kube-api-access-glzvz\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-etc-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.788315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-registration-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788319 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-os-release\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788342 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-conf-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788399 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-registration-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788431 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-daemon-config\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5htkw\" (UniqueName: \"kubernetes.io/projected/c14a3660-2a88-490e-b481-6a414a642760-kube-api-access-5htkw\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788483 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-host\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788511 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-socket-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788554 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788576 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-socket-dir-parent\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788653 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-etc-selinux\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788693 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-socket-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788731 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-etc-kubernetes\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788756 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-modprobe-d\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788778 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-sys\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788796 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c8296fea-0715-4071-8570-f4301f99baf7-agent-certs\") pod \"konnectivity-agent-72vbr\" (UID: \"c8296fea-0715-4071-8570-f4301f99baf7\") " pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788810 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788854 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788879 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad9aa6c3-01ba-48c8-a500-1468298afb49-iptables-alerter-script\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788903 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-kubelet\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788926 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysconfig\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788950 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.788980 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-cni-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789000 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysctl-d\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789019 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0303819b-d5e8-4964-94ed-2b90452a5036-hosts-file\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789041 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-hostroot\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789069 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-multus-certs\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-ovn\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789119 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-env-overrides\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789144 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789150 2557 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789170 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38712c2b-cfb9-4780-9720-5dcdbd748470-cni-binary-copy\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789196 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcbb4\" (UniqueName: \"kubernetes.io/projected/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-kube-api-access-jcbb4\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789229 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad9aa6c3-01ba-48c8-a500-1468298afb49-host-slash\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.789858 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789273 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-run-netns\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789300 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97tsf\" (UniqueName: \"kubernetes.io/projected/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-kube-api-access-97tsf\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789323 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/197197c7-6df9-4853-87c8-d67484c47710-etc-tuned\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789346 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-k8s-cni-cncf-io\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789371 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-netns\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789472 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-cni-multus\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789707 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ad9aa6c3-01ba-48c8-a500-1468298afb49-host-slash\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789742 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-lib-modules\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.790549 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.789930 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ad9aa6c3-01ba-48c8-a500-1468298afb49-iptables-alerter-script\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.792988 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.792966 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c8296fea-0715-4071-8570-f4301f99baf7-agent-certs\") pod \"konnectivity-agent-72vbr\" (UID: \"c8296fea-0715-4071-8570-f4301f99baf7\") " pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.800998 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.800974 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7dvp\" (UniqueName: \"kubernetes.io/projected/ad9aa6c3-01ba-48c8-a500-1468298afb49-kube-api-access-v7dvp\") pod \"iptables-alerter-wltjx\" (UID: \"ad9aa6c3-01ba-48c8-a500-1468298afb49\") " pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:14.803160 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.803141 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcbb4\" (UniqueName: \"kubernetes.io/projected/32c695b7-8c03-4026-a1c1-09b3efbcc3b2-kube-api-access-jcbb4\") pod \"aws-ebs-csi-driver-node-sdp46\" (UID: \"32c695b7-8c03-4026-a1c1-09b3efbcc3b2\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.874270 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.874225 2557 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 19:54:14.890891 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890854 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-k8s-cni-cncf-io\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890904 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-netns\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890920 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-cni-multus\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890935 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-k8s-cni-cncf-io\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890934 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-lib-modules\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890980 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-log-socket\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890987 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-netns\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.890996 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-cni-bin\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-system-cni-dir\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891024 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-cni-multus\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891053 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-cni-bin\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891064 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-log-socket\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-cnibin\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891092 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-system-cni-dir\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891116 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-dbus\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891120 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-lib-modules\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891140 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-var-lib-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891119 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-cnibin\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891167 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-var-lib-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891173 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-var-lib-kubelet\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891208 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/197197c7-6df9-4853-87c8-d67484c47710-tmp\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891225 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891270 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-dbus\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891294 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-var-lib-kubelet\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891326 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-kubelet\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.891346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891346 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-systemd-units\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891360 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-system-cni-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891375 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z8mzm\" (UniqueName: \"kubernetes.io/projected/38712c2b-cfb9-4780-9720-5dcdbd748470-kube-api-access-z8mzm\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891390 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-kubelet\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891389 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-kubelet-config\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.891391 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc31ca06-d978-4248-98fd-000e3fc287b0-ovn-node-metrics-cert\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891439 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-system-cni-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891447 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c14a3660-2a88-490e-b481-6a414a642760-serviceca\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.891480 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:54:15.391453713 +0000 UTC m=+3.062240234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891489 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-systemd-units\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891502 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-os-release\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891533 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891542 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-kubelet-config\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891560 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-cnibin\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891586 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-cni-netd\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891609 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-kubernetes\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.892035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891633 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-systemd\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891657 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-cni-bin\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891681 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891706 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttsh2\" (UniqueName: \"kubernetes.io/projected/fc31ca06-d978-4248-98fd-000e3fc287b0-kube-api-access-ttsh2\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891732 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ws6qj\" (UniqueName: \"kubernetes.io/projected/197197c7-6df9-4853-87c8-d67484c47710-kube-api-access-ws6qj\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891760 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-slash\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891789 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-systemd\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891814 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891840 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-run\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891887 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0303819b-d5e8-4964-94ed-2b90452a5036-tmp-dir\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891919 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891943 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-cni-binary-copy\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891970 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-ovnkube-config\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.891997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysctl-conf\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892020 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtn9s\" (UniqueName: \"kubernetes.io/projected/a9563bf9-bcc6-42c8-8a50-e54da8128432-kube-api-access-gtn9s\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-ovnkube-script-lib\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892047 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c14a3660-2a88-490e-b481-6a414a642760-serviceca\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.892832 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892091 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892122 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-node-log\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892154 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c14a3660-2a88-490e-b481-6a414a642760-host\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892205 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glzvz\" (UniqueName: \"kubernetes.io/projected/0303819b-d5e8-4964-94ed-2b90452a5036-kube-api-access-glzvz\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892247 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-etc-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892270 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-os-release\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892285 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-conf-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892301 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-daemon-config\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892321 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5htkw\" (UniqueName: \"kubernetes.io/projected/c14a3660-2a88-490e-b481-6a414a642760-kube-api-access-5htkw\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892341 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-host\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892359 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-socket-dir-parent\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892378 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-etc-kubernetes\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892395 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-modprobe-d\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892401 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-sys\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892464 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-kubelet\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892465 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-slash\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.893639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892483 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysconfig\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892501 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-systemd\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892536 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-cni-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892551 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysctl-d\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892555 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-systemd\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892585 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0303819b-d5e8-4964-94ed-2b90452a5036-hosts-file\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892589 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-cni-bin\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892626 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-hostroot\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892633 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/0303819b-d5e8-4964-94ed-2b90452a5036-hosts-file\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-multus-certs\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892663 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-conf-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892691 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-cni-binary-copy\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892746 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-ovn\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892770 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-hostroot\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892821 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-run-multus-certs\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892662 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysctl-d\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892868 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-host\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.894484 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892914 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-socket-dir-parent\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892949 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-etc-kubernetes\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892951 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0303819b-d5e8-4964-94ed-2b90452a5036-tmp-dir\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892693 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-run-ovn\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893016 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-modprobe-d\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893030 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-env-overrides\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893075 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-host-var-lib-kubelet\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893093 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysctl-conf\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893107 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38712c2b-cfb9-4780-9720-5dcdbd748470-cni-binary-copy\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893110 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-sysconfig\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893159 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-run-netns\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893172 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-node-log\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893193 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-97tsf\" (UniqueName: \"kubernetes.io/projected/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-kube-api-access-97tsf\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893216 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-run-ovn-kubernetes\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893218 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/197197c7-6df9-4853-87c8-d67484c47710-etc-tuned\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893263 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-tuning-conf-dir\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893325 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c14a3660-2a88-490e-b481-6a414a642760-host\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893327 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-cni-dir\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.893408 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.893474 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:54:15.393454383 +0000 UTC m=+3.064240887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893561 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-env-overrides\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893707 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-etc-openvswitch\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893054 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-sys\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893773 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-os-release\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.893780 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a9563bf9-bcc6-42c8-8a50-e54da8128432-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.892554 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a9563bf9-bcc6-42c8-8a50-e54da8128432-os-release\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894063 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894096 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-ovnkube-script-lib\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-etc-kubernetes\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894174 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-run-netns\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894218 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/fc31ca06-d978-4248-98fd-000e3fc287b0-host-cni-netd\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894272 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/197197c7-6df9-4853-87c8-d67484c47710-run\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894285 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/38712c2b-cfb9-4780-9720-5dcdbd748470-cnibin\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.895698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894346 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/38712c2b-cfb9-4780-9720-5dcdbd748470-cni-binary-copy\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.896304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894383 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/197197c7-6df9-4853-87c8-d67484c47710-tmp\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.896304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894457 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/38712c2b-cfb9-4780-9720-5dcdbd748470-multus-daemon-config\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.896304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.894639 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/fc31ca06-d978-4248-98fd-000e3fc287b0-ovnkube-config\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.896304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.896086 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/fc31ca06-d978-4248-98fd-000e3fc287b0-ovn-node-metrics-cert\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.896304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.896200 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/197197c7-6df9-4853-87c8-d67484c47710-etc-tuned\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.914368 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.914308 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:14.914368 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.914332 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:14.914368 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.914347 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:14.914531 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:14.914421 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:54:15.414404287 +0000 UTC m=+3.085190792 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:14.916527 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.916504 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ws6qj\" (UniqueName: \"kubernetes.io/projected/197197c7-6df9-4853-87c8-d67484c47710-kube-api-access-ws6qj\") pod \"tuned-kj66r\" (UID: \"197197c7-6df9-4853-87c8-d67484c47710\") " pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:14.926257 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.926216 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-glzvz\" (UniqueName: \"kubernetes.io/projected/0303819b-d5e8-4964-94ed-2b90452a5036-kube-api-access-glzvz\") pod \"node-resolver-4gk5v\" (UID: \"0303819b-d5e8-4964-94ed-2b90452a5036\") " pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:14.926351 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.926259 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-97tsf\" (UniqueName: \"kubernetes.io/projected/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-kube-api-access-97tsf\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:14.927301 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.927284 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttsh2\" (UniqueName: \"kubernetes.io/projected/fc31ca06-d978-4248-98fd-000e3fc287b0-kube-api-access-ttsh2\") pod \"ovnkube-node-gs6nt\" (UID: \"fc31ca06-d978-4248-98fd-000e3fc287b0\") " pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:14.929508 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.929486 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5htkw\" (UniqueName: \"kubernetes.io/projected/c14a3660-2a88-490e-b481-6a414a642760-kube-api-access-5htkw\") pod \"node-ca-5l4v9\" (UID: \"c14a3660-2a88-490e-b481-6a414a642760\") " pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:14.931639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.931617 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z8mzm\" (UniqueName: \"kubernetes.io/projected/38712c2b-cfb9-4780-9720-5dcdbd748470-kube-api-access-z8mzm\") pod \"multus-4vv77\" (UID: \"38712c2b-cfb9-4780-9720-5dcdbd748470\") " pod="openshift-multus/multus-4vv77" Jul 15 19:54:14.931721 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.931694 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtn9s\" (UniqueName: \"kubernetes.io/projected/a9563bf9-bcc6-42c8-8a50-e54da8128432-kube-api-access-gtn9s\") pod \"multus-additional-cni-plugins-ppbdn\" (UID: \"a9563bf9-bcc6-42c8-8a50-e54da8128432\") " pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:14.988510 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.988341 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:14.994309 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:14.994287 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" Jul 15 19:54:14.997696 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:14.997672 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8296fea_0715_4071_8570_f4301f99baf7.slice/crio-96412e33f4741706bdaec30a6ca96e81219bc9d9afa27fc32ee23b47c7fcddc9 WatchSource:0}: Error finding container 96412e33f4741706bdaec30a6ca96e81219bc9d9afa27fc32ee23b47c7fcddc9: Status 404 returned error can't find the container with id 96412e33f4741706bdaec30a6ca96e81219bc9d9afa27fc32ee23b47c7fcddc9 Jul 15 19:54:15.002658 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.002636 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wltjx" Jul 15 19:54:15.008768 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.008748 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" Jul 15 19:54:15.014983 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.014964 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4vv77" Jul 15 19:54:15.023855 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.022308 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:15.027287 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:15.027261 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38712c2b_cfb9_4780_9720_5dcdbd748470.slice/crio-e8d08bbb7d09029d08c704ea0350b92b095b689015af701815645eac3d041e93 WatchSource:0}: Error finding container e8d08bbb7d09029d08c704ea0350b92b095b689015af701815645eac3d041e93: Status 404 returned error can't find the container with id e8d08bbb7d09029d08c704ea0350b92b095b689015af701815645eac3d041e93 Jul 15 19:54:15.028857 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.028842 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-5l4v9" Jul 15 19:54:15.036060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.036042 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-kj66r" Jul 15 19:54:15.040066 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:15.040036 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc14a3660_2a88_490e_b481_6a414a642760.slice/crio-73c5c92671aa52b0559ebbb79ff53c25f80516d5b7e3b2693920fb3c4c2e55f3 WatchSource:0}: Error finding container 73c5c92671aa52b0559ebbb79ff53c25f80516d5b7e3b2693920fb3c4c2e55f3: Status 404 returned error can't find the container with id 73c5c92671aa52b0559ebbb79ff53c25f80516d5b7e3b2693920fb3c4c2e55f3 Jul 15 19:54:15.040759 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:15.040735 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc31ca06_d978_4248_98fd_000e3fc287b0.slice/crio-3508952c7b3b814c0611ca61d4e9fdd0f23fad0384f65ec1407365c56bdce68a WatchSource:0}: Error finding container 3508952c7b3b814c0611ca61d4e9fdd0f23fad0384f65ec1407365c56bdce68a: Status 404 returned error can't find the container with id 3508952c7b3b814c0611ca61d4e9fdd0f23fad0384f65ec1407365c56bdce68a Jul 15 19:54:15.041320 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.041296 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4gk5v" Jul 15 19:54:15.044038 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:15.044015 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod197197c7_6df9_4853_87c8_d67484c47710.slice/crio-4c8bfa12f5e8249afbe1327ca1f618f6c554fe9474f3277aebcdf4e9a2b7a5d1 WatchSource:0}: Error finding container 4c8bfa12f5e8249afbe1327ca1f618f6c554fe9474f3277aebcdf4e9a2b7a5d1: Status 404 returned error can't find the container with id 4c8bfa12f5e8249afbe1327ca1f618f6c554fe9474f3277aebcdf4e9a2b7a5d1 Jul 15 19:54:15.051552 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:15.051530 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0303819b_d5e8_4964_94ed_2b90452a5036.slice/crio-f72bd00fa89c4e97e32c5707be0522c1012fa456ff57a44d92be41414be9f617 WatchSource:0}: Error finding container f72bd00fa89c4e97e32c5707be0522c1012fa456ff57a44d92be41414be9f617: Status 404 returned error can't find the container with id f72bd00fa89c4e97e32c5707be0522c1012fa456ff57a44d92be41414be9f617 Jul 15 19:54:15.397818 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.397688 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:15.397818 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.397748 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:15.398010 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.397860 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:15.398010 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.397925 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:54:16.397905763 +0000 UTC m=+4.068692265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:15.398010 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.397859 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:15.398170 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.398037 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:54:16.39801453 +0000 UTC m=+4.068801044 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:15.498883 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.498843 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:15.499076 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.499025 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:15.499076 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.499050 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:15.499076 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.499062 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:15.499230 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:15.499127 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:54:16.499105591 +0000 UTC m=+4.169892097 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:15.781524 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.781409 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:49:14 +0000 UTC" deadline="2028-03-11 22:17:47.668898625 +0000 UTC" Jul 15 19:54:15.781524 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.781473 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14522h23m31.887429737s" Jul 15 19:54:15.815255 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.814616 2557 generic.go:356] "Generic (PLEG): container finished" podID="5dfe48926f4c41d3cac7cad190b9f419" containerID="088f2348c07a75ad8f74f04bd48e1622cbc11bd5b911a1009d8d5b7ccd552790" exitCode=0 Jul 15 19:54:15.815255 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.814727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" event={"ID":"5dfe48926f4c41d3cac7cad190b9f419","Type":"ContainerDied","Data":"088f2348c07a75ad8f74f04bd48e1622cbc11bd5b911a1009d8d5b7ccd552790"} Jul 15 19:54:15.834860 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.834395 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5l4v9" event={"ID":"c14a3660-2a88-490e-b481-6a414a642760","Type":"ContainerStarted","Data":"73c5c92671aa52b0559ebbb79ff53c25f80516d5b7e3b2693920fb3c4c2e55f3"} Jul 15 19:54:15.853190 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.853083 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4vv77" event={"ID":"38712c2b-cfb9-4780-9720-5dcdbd748470","Type":"ContainerStarted","Data":"e8d08bbb7d09029d08c704ea0350b92b095b689015af701815645eac3d041e93"} Jul 15 19:54:15.873411 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.873366 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerStarted","Data":"96582637ccad00175136b349ba4d23762731e7d8b7406fd5635bd1cdf290740d"} Jul 15 19:54:15.889678 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.889645 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" event={"ID":"32c695b7-8c03-4026-a1c1-09b3efbcc3b2","Type":"ContainerStarted","Data":"29c5b34cb5cc41db1a51b5999e677c09c9a07adb647353d5bce4f8ceb027e1fc"} Jul 15 19:54:15.910364 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.910325 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4gk5v" event={"ID":"0303819b-d5e8-4964-94ed-2b90452a5036","Type":"ContainerStarted","Data":"f72bd00fa89c4e97e32c5707be0522c1012fa456ff57a44d92be41414be9f617"} Jul 15 19:54:15.925125 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.925081 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kj66r" event={"ID":"197197c7-6df9-4853-87c8-d67484c47710","Type":"ContainerStarted","Data":"4c8bfa12f5e8249afbe1327ca1f618f6c554fe9474f3277aebcdf4e9a2b7a5d1"} Jul 15 19:54:15.933113 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.933010 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"3508952c7b3b814c0611ca61d4e9fdd0f23fad0384f65ec1407365c56bdce68a"} Jul 15 19:54:15.952850 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.952809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wltjx" event={"ID":"ad9aa6c3-01ba-48c8-a500-1468298afb49","Type":"ContainerStarted","Data":"a094ccbcbb858731bb8c213d170d727f3346e74261a2d4037cbe3c67468cc7fd"} Jul 15 19:54:15.970075 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:15.969986 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-72vbr" event={"ID":"c8296fea-0715-4071-8570-f4301f99baf7","Type":"ContainerStarted","Data":"96412e33f4741706bdaec30a6ca96e81219bc9d9afa27fc32ee23b47c7fcddc9"} Jul 15 19:54:16.406689 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.406594 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:16.406689 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.406678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:16.406953 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.406822 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:16.406953 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.406891 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:54:18.40686664 +0000 UTC m=+6.077653149 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:16.407309 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.407289 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:16.407385 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.407347 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:54:18.407330929 +0000 UTC m=+6.078117434 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:16.508571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.507362 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:16.508571 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.507588 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:16.508571 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.507612 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:16.508571 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.507626 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:16.508571 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.507690 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:54:18.507670751 +0000 UTC m=+6.178457274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:16.781599 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.781515 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:16.782121 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.781667 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:16.782121 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.782097 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:16.782248 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.782200 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:16.782989 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.782342 2557 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 19:49:14 +0000 UTC" deadline="2028-03-02 10:11:36.501835776 +0000 UTC" Jul 15 19:54:16.782989 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.782409 2557 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14294h17m19.719431384s" Jul 15 19:54:16.784641 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:16.784164 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:16.784641 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:16.784296 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:17.008498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:17.008460 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" event={"ID":"5dfe48926f4c41d3cac7cad190b9f419","Type":"ContainerStarted","Data":"1882747d03ce99c819164ba58a5bb37314158d0699a41b9ca5d59f9c4990580c"} Jul 15 19:54:18.424011 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:18.423978 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:18.424427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:18.424039 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:18.424427 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.424171 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:18.424427 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.424230 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:54:22.424207744 +0000 UTC m=+10.094994249 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:18.424644 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.424623 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:18.424733 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.424692 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:54:22.424676233 +0000 UTC m=+10.095462735 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:18.524667 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:18.524538 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:18.524836 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.524739 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:18.524836 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.524765 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:18.524836 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.524778 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:18.524936 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.524840 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:54:22.524823885 +0000 UTC m=+10.195610387 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:18.782368 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:18.781626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:18.782368 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.781759 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:18.782368 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:18.782095 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:18.782368 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.782187 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:18.782368 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:18.782219 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:18.782368 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:18.782294 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:20.779947 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:20.779917 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:20.779947 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:20.779936 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:20.780397 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:20.779920 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:20.780397 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:20.780102 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:20.780397 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:20.780261 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:20.780397 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:20.780367 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:22.463627 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:22.463556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:22.464044 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:22.463666 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:22.464044 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.463789 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:22.464044 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.463851 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:54:30.463830965 +0000 UTC m=+18.134617470 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:22.464312 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.464288 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:22.464389 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.464345 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:54:30.464329894 +0000 UTC m=+18.135116404 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:22.565393 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:22.564704 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:22.565393 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.564910 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:22.565393 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.564935 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:22.565393 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.564949 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:22.565393 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.565016 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:54:30.564996841 +0000 UTC m=+18.235783358 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:22.780891 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:22.780805 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:22.781061 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:22.780906 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:22.781061 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.780928 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:22.781061 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:22.780954 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:22.781217 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.781059 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:22.781217 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:22.781121 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:24.780440 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:24.780354 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:24.780440 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:24.780415 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:24.780440 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:24.780428 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:24.780970 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:24.780558 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:24.780970 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:24.780632 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:24.780970 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:24.780726 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:26.779530 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:26.779494 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:26.779911 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:26.779500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:26.779911 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:26.779626 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:26.779911 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:26.779629 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:26.779911 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:26.779735 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:26.779911 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:26.779833 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:28.779456 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:28.779421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:28.779911 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:28.779421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:28.779911 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:28.779569 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:28.779911 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:28.779421 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:28.779911 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:28.779614 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:28.779911 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:28.779722 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:30.524524 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:30.524486 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:30.524972 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:30.524544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:30.524972 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.524647 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:30.524972 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.524647 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:30.524972 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.524704 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:54:46.524688411 +0000 UTC m=+34.195474915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:30.524972 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.524719 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:54:46.52471271 +0000 UTC m=+34.195499212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:30.625753 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:30.625720 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:30.625895 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.625862 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:30.625895 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.625884 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:30.625895 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.625896 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:30.626010 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.625946 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:54:46.625930881 +0000 UTC m=+34.296717384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:30.780263 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:30.779651 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:30.780263 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:30.779651 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:30.780263 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.779791 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:30.780263 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.779860 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:30.780263 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:30.779876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:30.780263 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:30.779971 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:32.780514 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:32.780475 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:32.780981 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:32.780598 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:32.780981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:32.780972 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:32.781067 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:32.781039 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:32.781274 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:32.781084 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:32.781274 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:32.781144 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:34.779977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:34.779946 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:34.779977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:34.779970 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:34.780608 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:34.779946 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:34.780608 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:34.780069 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:34.780608 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:34.780174 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:34.780608 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:34.780283 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:36.051518 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.051293 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4gk5v" event={"ID":"0303819b-d5e8-4964-94ed-2b90452a5036","Type":"ContainerStarted","Data":"39111ac2997eba0ee019cd2ce65254c81c8290dd2fc647f9a8dc82dc2509ccff"} Jul 15 19:54:36.053052 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.053015 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-kj66r" event={"ID":"197197c7-6df9-4853-87c8-d67484c47710","Type":"ContainerStarted","Data":"cbf2732a2bb1b6821929915e2e818918403df4ae1b05b1efed5de60896003a86"} Jul 15 19:54:36.054588 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.054564 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-72vbr" event={"ID":"c8296fea-0715-4071-8570-f4301f99baf7","Type":"ContainerStarted","Data":"eed5464956b6b86b9883dc37bb1df285f58a8ad0776b0ead5fbfaa8b9fe2df7d"} Jul 15 19:54:36.056134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.056111 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" event={"ID":"a843563177c22b96d13b44f8b4e55822","Type":"ContainerStarted","Data":"ea8c13c4112c567ce22cb2a39104cb86c0f59537683f44b608da1ba64ce9cd5a"} Jul 15 19:54:36.057614 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.057591 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-5l4v9" event={"ID":"c14a3660-2a88-490e-b481-6a414a642760","Type":"ContainerStarted","Data":"c1b2e3fea86745ef42e2f434bd25438c2963e199d402eee0fe2999e172dc8a2f"} Jul 15 19:54:36.059094 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.059076 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4vv77" event={"ID":"38712c2b-cfb9-4780-9720-5dcdbd748470","Type":"ContainerStarted","Data":"24857fcecd9f40f8719663ecd402ad49dce03d92dfd98e462236111d9a413e98"} Jul 15 19:54:36.060654 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.060623 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9563bf9-bcc6-42c8-8a50-e54da8128432" containerID="bb92e8107eabbadb67a377da93770462b1cb08ceddd27aa86d94a5eff55e8001" exitCode=0 Jul 15 19:54:36.060740 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.060702 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerDied","Data":"bb92e8107eabbadb67a377da93770462b1cb08ceddd27aa86d94a5eff55e8001"} Jul 15 19:54:36.065105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.065017 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" event={"ID":"32c695b7-8c03-4026-a1c1-09b3efbcc3b2","Type":"ContainerStarted","Data":"b37f1b85a396c7931efaef325dfc63503e803ed24f0a8f2d2cbf39ea603727fe"} Jul 15 19:54:36.069398 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.069349 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-12.ec2.internal" podStartSLOduration=22.069337081 podStartE2EDuration="22.069337081s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:17.033054118 +0000 UTC m=+4.703840644" watchObservedRunningTime="2026-07-15 19:54:36.069337081 +0000 UTC m=+23.740123608" Jul 15 19:54:36.086557 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.086513 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4gk5v" podStartSLOduration=3.14047092 podStartE2EDuration="23.086498975s" totalImagesPullingTime="19.946028055s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:36.069333397 +0000 UTC m=+23.740119942" watchObservedRunningTime="2026-07-15 19:54:36.086498975 +0000 UTC m=+23.757285554" Jul 15 19:54:36.086851 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.086822 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-72vbr" podStartSLOduration=4.087453523 podStartE2EDuration="24.08681656s" totalImagesPullingTime="19.999363037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:36.086157446 +0000 UTC m=+23.756943971" watchObservedRunningTime="2026-07-15 19:54:36.08681656 +0000 UTC m=+23.757603085" Jul 15 19:54:36.146672 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.146623 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-kj66r" podStartSLOduration=3.057191042 podStartE2EDuration="23.146602908s" totalImagesPullingTime="20.089411866s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:36.146427366 +0000 UTC m=+23.817213893" watchObservedRunningTime="2026-07-15 19:54:36.146602908 +0000 UTC m=+23.817389434" Jul 15 19:54:36.147068 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.147027 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-5l4v9" podStartSLOduration=3.07576664 podStartE2EDuration="23.147018075s" totalImagesPullingTime="20.071251435s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:36.127006163 +0000 UTC m=+23.797792691" watchObservedRunningTime="2026-07-15 19:54:36.147018075 +0000 UTC m=+23.817804601" Jul 15 19:54:36.166077 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.166039 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-12.ec2.internal" podStartSLOduration=22.16602571 podStartE2EDuration="22.16602571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:36.165559674 +0000 UTC m=+23.836346198" watchObservedRunningTime="2026-07-15 19:54:36.16602571 +0000 UTC m=+23.836812236" Jul 15 19:54:36.667164 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.666936 2557 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 19:54:36.745692 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.745603 2557 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T19:54:36.667146669Z","UUID":"fcaf6313-77d7-4d06-99e4-cab0e71e75d5","Handler":null,"Name":"","Endpoint":""} Jul 15 19:54:36.747473 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.747451 2557 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 19:54:36.747568 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.747512 2557 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 19:54:36.780114 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.780081 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:36.780114 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.780095 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:36.780306 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:36.780175 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:36.780306 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:36.780227 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:36.780421 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:36.780344 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:36.780478 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:36.780422 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:37.069844 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.069810 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" event={"ID":"32c695b7-8c03-4026-a1c1-09b3efbcc3b2","Type":"ContainerStarted","Data":"e1c977eb20f3e6a96e5576c069bd8aadc3524e9194f377b1aa94ec8d9df7e3c3"} Jul 15 19:54:37.073689 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.073661 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"8a9d26354786c5165b3de632a470f690841a89c6bb55b8d71d5bdf1787bdd2e8"} Jul 15 19:54:37.073793 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.073747 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"93c431d9a571fe8dcadac5cc8840cbdee58a5689eefe69f92d2fcb9e90be3de4"} Jul 15 19:54:37.073861 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.073796 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"d95bf758449a6df9fb719c7a39d8d1b9f0699afa5d813f2d0febd0e7e51aa64d"} Jul 15 19:54:37.073861 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.073843 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"8ea9ebd8014d0dea641d599533c9d56c06e9dc70569bc3838c13a5f78cb52b80"} Jul 15 19:54:37.073971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.073891 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"e54e6c6d31c7a86bd595081c793b3cbe5389759df4bc95d8b7d9ff840446348c"} Jul 15 19:54:37.073971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.073941 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"1d487c6eacaa7393af476a51e630046c73a9dd8324376285570240a9296ab770"} Jul 15 19:54:37.075398 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.075362 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wltjx" event={"ID":"ad9aa6c3-01ba-48c8-a500-1468298afb49","Type":"ContainerStarted","Data":"06cdfc28b4c4ee1a8acfad7e4f6efcf8399ed174dc06b2fce5d40a18e3946ef9"} Jul 15 19:54:37.095253 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.095188 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-4vv77" podStartSLOduration=3.957926175 podStartE2EDuration="24.095172979s" totalImagesPullingTime="20.137246804s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:36.190972331 +0000 UTC m=+23.861758856" watchObservedRunningTime="2026-07-15 19:54:37.095172979 +0000 UTC m=+24.765959505" Jul 15 19:54:37.095415 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:37.095394 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wltjx" podStartSLOduration=5.118521901 podStartE2EDuration="25.095387638s" totalImagesPullingTime="19.976865737s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:37.094716365 +0000 UTC m=+24.765502889" watchObservedRunningTime="2026-07-15 19:54:37.095387638 +0000 UTC m=+24.766174163" Jul 15 19:54:38.780362 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:38.780055 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:38.780812 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:38.780149 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:38.780812 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:38.780432 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:38.780812 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:38.780149 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:38.780812 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:38.780493 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:38.780812 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:38.780547 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:39.082923 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:39.082824 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" event={"ID":"32c695b7-8c03-4026-a1c1-09b3efbcc3b2","Type":"ContainerStarted","Data":"df1f6e08b3dbcef46b17b5a634eb87d693b2df1cbd84e8b14fc4e69fa80d9dde"} Jul 15 19:54:39.103505 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:39.103454 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sdp46" podStartSLOduration=4.08695839 podStartE2EDuration="27.103438523s" totalImagesPullingTime="23.016480133s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:12 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:39.103328959 +0000 UTC m=+26.774115485" watchObservedRunningTime="2026-07-15 19:54:39.103438523 +0000 UTC m=+26.774225047" Jul 15 19:54:40.252597 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:40.252576 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:40.256450 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:40.256377 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:40.779474 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:40.779442 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:40.779617 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:40.779443 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:40.779617 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:40.779550 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:40.779617 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:40.779568 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:40.779786 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:40.779645 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:40.779786 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:40.779752 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:41.088971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:41.088942 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9563bf9-bcc6-42c8-8a50-e54da8128432" containerID="0eff2ed71bf233d4fdf9b5e24843d4b885a4a1ceb4e6ee51cd2d222bc7ad69e7" exitCode=0 Jul 15 19:54:41.089159 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:41.089026 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerDied","Data":"0eff2ed71bf233d4fdf9b5e24843d4b885a4a1ceb4e6ee51cd2d222bc7ad69e7"} Jul 15 19:54:41.092556 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:41.092532 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"9c273cc9f6cfa7dd65d1d014aedcb7e3391303dc60aa1062073f15ba253fa281"} Jul 15 19:54:41.092816 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:41.092760 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:41.093310 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:41.093292 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-72vbr" Jul 15 19:54:42.781660 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:42.781498 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:42.782226 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:42.781551 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:42.782226 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:42.781723 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:42.782226 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:42.781561 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:42.782226 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:42.781792 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:42.782226 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:42.781903 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:43.099287 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.099168 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" event={"ID":"fc31ca06-d978-4248-98fd-000e3fc287b0","Type":"ContainerStarted","Data":"e4dca00a29a810e87866adf28d63586690cf53708da950a104ceffda24aeb031"} Jul 15 19:54:43.099454 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.099431 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:43.099508 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.099490 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:43.099546 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.099538 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:43.101459 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.101434 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9563bf9-bcc6-42c8-8a50-e54da8128432" containerID="176d102aa9b9bf0af2f0e99a7548dd7d02dc6b376a424fef63eb45e384af24af" exitCode=0 Jul 15 19:54:43.101583 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.101524 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerDied","Data":"176d102aa9b9bf0af2f0e99a7548dd7d02dc6b376a424fef63eb45e384af24af"} Jul 15 19:54:43.116504 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.116483 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:43.116614 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.116580 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:54:43.145106 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:43.145064 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" podStartSLOduration=9.005979645 podStartE2EDuration="30.145027421s" totalImagesPullingTime="21.139047776s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:43.14377413 +0000 UTC m=+30.814560655" watchObservedRunningTime="2026-07-15 19:54:43.145027421 +0000 UTC m=+30.815813943" Jul 15 19:54:44.782958 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:44.782927 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:44.783533 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:44.783027 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:44.783533 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:44.783036 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:44.783533 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:44.783053 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:44.783533 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:44.783129 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:44.783533 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:44.783199 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:45.107537 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.107457 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9563bf9-bcc6-42c8-8a50-e54da8128432" containerID="60fc1e77bbae2b0b41c9acbbca569a5836472580aa7baf6ec625ffbeccb19dba" exitCode=0 Jul 15 19:54:45.107697 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.107533 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerDied","Data":"60fc1e77bbae2b0b41c9acbbca569a5836472580aa7baf6ec625ffbeccb19dba"} Jul 15 19:54:45.213731 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.213700 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ftj7m"] Jul 15 19:54:45.213922 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.213838 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:45.213975 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:45.213942 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:45.219124 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.219099 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-45bw2"] Jul 15 19:54:45.219247 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.219175 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:45.219303 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:45.219267 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:45.222024 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.221992 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tr84l"] Jul 15 19:54:45.222145 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:45.222087 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:45.222206 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:45.222176 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:46.539712 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:46.539668 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:46.540170 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:46.539763 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:46.540170 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.539834 2557 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:46.540170 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.539879 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:46.540170 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.539929 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret podName:4e54a4d4-ffa7-4e74-8cad-960dceb020ab nodeName:}" failed. No retries permitted until 2026-07-15 19:55:18.539883401 +0000 UTC m=+66.210669903 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret") pod "global-pull-secret-syncer-tr84l" (UID: "4e54a4d4-ffa7-4e74-8cad-960dceb020ab") : object "kube-system"/"original-pull-secret" not registered Jul 15 19:54:46.540170 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.539954 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:55:18.539942926 +0000 UTC m=+66.210729429 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 19:54:46.641042 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:46.641011 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:46.641262 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.641152 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 19:54:46.641262 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.641174 2557 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 19:54:46.641262 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.641185 2557 projected.go:196] Error preparing data for projected volume kube-api-access-2mkmd for pod openshift-network-diagnostics/network-check-target-45bw2: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:46.641262 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.641259 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd podName:bb263d27-b8f9-4b8c-8297-a584a6742fdb nodeName:}" failed. No retries permitted until 2026-07-15 19:55:18.641222983 +0000 UTC m=+66.312009493 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-2mkmd" (UniqueName: "kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd") pod "network-check-target-45bw2" (UID: "bb263d27-b8f9-4b8c-8297-a584a6742fdb") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 19:54:46.782919 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:46.782840 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:46.783068 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:46.782947 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:46.783068 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.782959 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:46.783068 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:46.782974 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:46.783068 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.783057 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:46.783217 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:46.783142 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:48.782277 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:48.782224 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:48.782654 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:48.782355 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:48.782654 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:48.782368 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-45bw2" podUID="bb263d27-b8f9-4b8c-8297-a584a6742fdb" Jul 15 19:54:48.782654 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:48.782380 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:48.782654 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:48.782453 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tr84l" podUID="4e54a4d4-ffa7-4e74-8cad-960dceb020ab" Jul 15 19:54:48.782654 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:48.782540 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-ftj7m" podUID="2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed" Jul 15 19:54:49.186017 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.185990 2557 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-12.ec2.internal" event="NodeReady" Jul 15 19:54:49.186134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.186117 2557 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 19:54:49.241896 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.241866 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7f77bc4dff-4k6n2"] Jul 15 19:54:49.243588 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.243572 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.246850 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.246827 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pndwd\"" Jul 15 19:54:49.247678 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.247658 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 19:54:49.247776 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.247678 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 19:54:49.248736 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.248711 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 19:54:49.255051 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.255030 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 19:54:49.280538 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.279768 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f77bc4dff-4k6n2"] Jul 15 19:54:49.282442 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.282414 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-f7wqb"] Jul 15 19:54:49.284545 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.284526 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.287958 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.287937 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 19:54:49.288177 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.288162 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 19:54:49.288689 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.288672 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c5qnq\"" Jul 15 19:54:49.291199 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.291178 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-blcpq"] Jul 15 19:54:49.292773 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.292756 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.295784 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.295768 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 19:54:49.295918 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.295895 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f7wqb"] Jul 15 19:54:49.296030 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.295800 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-zbcvf\"" Jul 15 19:54:49.296030 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.295799 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 19:54:49.296030 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.295844 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 19:54:49.310636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.310595 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:49.321122 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.321098 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-blcpq"] Jul 15 19:54:49.361090 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361066 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-image-registry-private-configuration\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361210 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bb996b87-aef6-49fd-b396-1a91c740b141-ca-trust-extracted\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361210 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361111 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-bound-sa-token\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361210 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361178 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2trl6\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-kube-api-access-2trl6\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361337 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-registry-certificates\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361337 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361260 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361337 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361285 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-trusted-ca\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.361337 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.361328 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-installation-pull-secrets\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.431865 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.431840 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:49.461950 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.461920 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462086 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.461964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f9jcd\" (UniqueName: \"kubernetes.io/projected/1f70d4bd-946c-415a-b6c7-6546d7760288-kube-api-access-f9jcd\") pod \"ingress-canary-blcpq\" (UID: \"1f70d4bd-946c-415a-b6c7-6546d7760288\") " pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.462086 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.461992 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-trusted-ca\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462086 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462018 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-installation-pull-secrets\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462086 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.462069 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:54:49.462086 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462075 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-image-registry-private-configuration\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462102 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bb996b87-aef6-49fd-b396-1a91c740b141-ca-trust-extracted\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462129 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-bound-sa-token\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462155 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2602891b-0d4f-4354-964e-c028e469d84b-tmp-dir\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.462084 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462210 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2trl6\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-kube-api-access-2trl6\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462258 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2602891b-0d4f-4354-964e-c028e469d84b-config-volume\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.462278 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:49.962229958 +0000 UTC m=+37.633016460 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:54:49.462373 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462344 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktl5d\" (UniqueName: \"kubernetes.io/projected/2602891b-0d4f-4354-964e-c028e469d84b-kube-api-access-ktl5d\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.462693 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462385 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f70d4bd-946c-415a-b6c7-6546d7760288-cert\") pod \"ingress-canary-blcpq\" (UID: \"1f70d4bd-946c-415a-b6c7-6546d7760288\") " pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.462693 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462421 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.462693 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462468 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-registry-certificates\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462807 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462791 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bb996b87-aef6-49fd-b396-1a91c740b141-ca-trust-extracted\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.462957 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.462939 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-registry-certificates\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.463054 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.463035 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-trusted-ca\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.467582 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.467555 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-image-registry-private-configuration\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.467684 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.467618 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-installation-pull-secrets\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.473725 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.473696 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-bound-sa-token\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.473816 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.473788 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2trl6\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-kube-api-access-2trl6\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.510545 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.510511 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:49.563745 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.563714 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f9jcd\" (UniqueName: \"kubernetes.io/projected/1f70d4bd-946c-415a-b6c7-6546d7760288-kube-api-access-f9jcd\") pod \"ingress-canary-blcpq\" (UID: \"1f70d4bd-946c-415a-b6c7-6546d7760288\") " pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.563884 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.563768 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2602891b-0d4f-4354-964e-c028e469d84b-tmp-dir\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.563990 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.563968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2602891b-0d4f-4354-964e-c028e469d84b-config-volume\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.564025 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.564002 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktl5d\" (UniqueName: \"kubernetes.io/projected/2602891b-0d4f-4354-964e-c028e469d84b-kube-api-access-ktl5d\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.564058 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.564024 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f70d4bd-946c-415a-b6c7-6546d7760288-cert\") pod \"ingress-canary-blcpq\" (UID: \"1f70d4bd-946c-415a-b6c7-6546d7760288\") " pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.564058 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.564052 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.564140 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.564123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2602891b-0d4f-4354-964e-c028e469d84b-tmp-dir\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.564182 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.564157 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:54:49.564226 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.564220 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:54:50.064206094 +0000 UTC m=+37.734992622 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:54:49.564555 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.564537 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2602891b-0d4f-4354-964e-c028e469d84b-config-volume\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.566519 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.566504 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1f70d4bd-946c-415a-b6c7-6546d7760288-cert\") pod \"ingress-canary-blcpq\" (UID: \"1f70d4bd-946c-415a-b6c7-6546d7760288\") " pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.583609 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.583583 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f9jcd\" (UniqueName: \"kubernetes.io/projected/1f70d4bd-946c-415a-b6c7-6546d7760288-kube-api-access-f9jcd\") pod \"ingress-canary-blcpq\" (UID: \"1f70d4bd-946c-415a-b6c7-6546d7760288\") " pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.584937 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.584917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktl5d\" (UniqueName: \"kubernetes.io/projected/2602891b-0d4f-4354-964e-c028e469d84b-kube-api-access-ktl5d\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:49.602970 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.602950 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-blcpq" Jul 15 19:54:49.621684 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.621657 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:49.750220 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.750019 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-blcpq"] Jul 15 19:54:49.760786 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:54:49.760759 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f70d4bd_946c_415a_b6c7_6546d7760288.slice/crio-3a934dd8348debeb05727b98857b30a8640abef56c40e143ba4e57d1fbc7ac6a WatchSource:0}: Error finding container 3a934dd8348debeb05727b98857b30a8640abef56c40e143ba4e57d1fbc7ac6a: Status 404 returned error can't find the container with id 3a934dd8348debeb05727b98857b30a8640abef56c40e143ba4e57d1fbc7ac6a Jul 15 19:54:49.890523 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.890493 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:49.968777 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.968656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:49.968914 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.968819 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:54:49.968914 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.968834 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:54:49.968914 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:49.968902 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:50.968882796 +0000 UTC m=+38.639669301 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:54:49.992824 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:49.992794 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:50.069979 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.069945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:50.070140 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:50.070102 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:54:50.070184 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:50.070169 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:54:51.070151928 +0000 UTC m=+38.740938445 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:54:50.070815 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.070784 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:50.121477 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.121445 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9563bf9-bcc6-42c8-8a50-e54da8128432" containerID="7b17007f3ecec69d7ee1f461c366fced77d17131c8dc353551af7008b4ba5090" exitCode=0 Jul 15 19:54:50.121631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.121519 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerDied","Data":"7b17007f3ecec69d7ee1f461c366fced77d17131c8dc353551af7008b4ba5090"} Jul 15 19:54:50.122736 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.122715 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-blcpq" event={"ID":"1f70d4bd-946c-415a-b6c7-6546d7760288","Type":"ContainerStarted","Data":"3a934dd8348debeb05727b98857b30a8640abef56c40e143ba4e57d1fbc7ac6a"} Jul 15 19:54:50.669716 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.669626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:50.779695 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.779654 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:54:50.779896 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.779654 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:54:50.779896 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.779654 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:54:50.783479 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.783459 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:54:50.783679 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.783664 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:54:50.784731 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.784705 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:54:50.788197 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.785147 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjg99\"" Jul 15 19:54:50.788197 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.785212 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q7pb7\"" Jul 15 19:54:50.788197 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.785466 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:54:50.789645 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.789624 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:50.977615 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:50.977564 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:50.978063 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:50.977758 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:54:50.978063 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:50.977779 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:54:50.978063 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:50.977863 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:52.97784304 +0000 UTC m=+40.648629544 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:54:51.078560 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:51.078521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:51.078708 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:51.078675 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:54:51.078765 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:51.078752 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:54:53.078736425 +0000 UTC m=+40.749522928 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:54:51.129941 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:51.129905 2557 generic.go:356] "Generic (PLEG): container finished" podID="a9563bf9-bcc6-42c8-8a50-e54da8128432" containerID="60ea0e3867e27f7c73bc7006523ac7d2bb3f10446fcf67fe59c26289297450d0" exitCode=0 Jul 15 19:54:51.130122 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:51.129979 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerDied","Data":"60ea0e3867e27f7c73bc7006523ac7d2bb3f10446fcf67fe59c26289297450d0"} Jul 15 19:54:52.137626 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:52.137596 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" event={"ID":"a9563bf9-bcc6-42c8-8a50-e54da8128432","Type":"ContainerStarted","Data":"b83af667f1fa6ab0d32af03ecaa2694ad5c440976a70de93e2af6756bcd6f951"} Jul 15 19:54:52.219639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:52.219584 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-ppbdn" podStartSLOduration=9.788450576 podStartE2EDuration="40.219565784s" totalImagesPullingTime="30.431115208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:12 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:52.217898524 +0000 UTC m=+39.888685048" watchObservedRunningTime="2026-07-15 19:54:52.219565784 +0000 UTC m=+39.890352338" Jul 15 19:54:52.423610 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:52.423523 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:52.526510 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:52.526480 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:52.632406 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:52.632370 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:52.994397 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:52.994305 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:52.994542 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:52.994455 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:54:52.994542 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:52.994469 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:54:52.994542 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:52.994526 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:54:56.994509585 +0000 UTC m=+44.665296088 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:54:53.097682 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:53.095224 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:53.097682 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:53.095428 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:54:53.097682 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:53.095503 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:54:57.095482743 +0000 UTC m=+44.766269266 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:54:53.141226 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:53.141183 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-blcpq" event={"ID":"1f70d4bd-946c-415a-b6c7-6546d7760288","Type":"ContainerStarted","Data":"d276fdcd1a8552de3bdcfd0be94cbf404a9f89cf83273d11558eaa42b05c7739"} Jul 15 19:54:53.170469 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:53.170421 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-blcpq" podStartSLOduration=1.496352246 podStartE2EDuration="4.170405901s" totalImagesPullingTime="2.674053655s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:54:53.169092953 +0000 UTC m=+40.839879478" watchObservedRunningTime="2026-07-15 19:54:53.170405901 +0000 UTC m=+40.841192447" Jul 15 19:54:53.640329 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:53.640294 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:57.024275 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:57.024216 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:54:57.024693 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:57.024353 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:54:57.024693 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:57.024373 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:54:57.024693 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:57.024448 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:05.024431105 +0000 UTC m=+52.695217608 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:54:57.124797 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:57.124760 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:54:57.124985 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:57.124921 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:54:57.125025 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:54:57.124988 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:55:05.124968376 +0000 UTC m=+52.795754879 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:54:58.113620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:58.113584 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:54:59.572059 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:54:59.572021 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:02.354173 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:02.354138 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:02.454517 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:02.454466 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:02.541143 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:02.541070 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:03.724607 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:03.724575 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:04.807273 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:04.807220 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:04.905997 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:04.905966 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:05.083070 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:05.082982 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:55:05.083229 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:05.083127 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:55:05.083229 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:05.083146 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:55:05.083229 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:05.083214 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:21.083197933 +0000 UTC m=+68.753984436 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:55:05.184228 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:05.184195 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:55:05.184402 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:05.184318 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:55:05.184402 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:05.184367 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:55:21.184353919 +0000 UTC m=+68.855140422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:55:07.891895 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:07.891858 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:07.999213 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:07.999184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:15.118637 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:15.118611 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-gs6nt" Jul 15 19:55:15.339433 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:15.339392 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:15.426833 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:15.426808 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:18.570963 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.570912 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:55:18.571422 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.571016 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:55:18.574277 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.574255 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 19:55:18.574533 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.574517 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 19:55:18.581105 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:18.581089 2557 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 19:55:18.581193 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:18.581164 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs podName:2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed nodeName:}" failed. No retries permitted until 2026-07-15 19:56:22.581142097 +0000 UTC m=+130.251928601 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs") pod "network-metrics-daemon-ftj7m" (UID: "2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed") : secret "metrics-daemon-secret" not found Jul 15 19:55:18.584781 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.584761 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4e54a4d4-ffa7-4e74-8cad-960dceb020ab-original-pull-secret\") pod \"global-pull-secret-syncer-tr84l\" (UID: \"4e54a4d4-ffa7-4e74-8cad-960dceb020ab\") " pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:55:18.672114 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.672068 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:55:18.675122 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.675100 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 19:55:18.685728 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.685705 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 19:55:18.695756 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.695730 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2mkmd\" (UniqueName: \"kubernetes.io/projected/bb263d27-b8f9-4b8c-8297-a584a6742fdb-kube-api-access-2mkmd\") pod \"network-check-target-45bw2\" (UID: \"bb263d27-b8f9-4b8c-8297-a584a6742fdb\") " pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:55:18.706473 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.706449 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tr84l" Jul 15 19:55:18.853406 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:18.853326 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tr84l"] Jul 15 19:55:18.871892 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:18.871857 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e54a4d4_ffa7_4e74_8cad_960dceb020ab.slice/crio-346e8f635503a96b4b04f009380a80af56b43a4fc4d71473ce3884b8a7ab36d9 WatchSource:0}: Error finding container 346e8f635503a96b4b04f009380a80af56b43a4fc4d71473ce3884b8a7ab36d9: Status 404 returned error can't find the container with id 346e8f635503a96b4b04f009380a80af56b43a4fc4d71473ce3884b8a7ab36d9 Jul 15 19:55:19.001788 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:19.001753 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-q7pb7\"" Jul 15 19:55:19.004836 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:19.004818 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:55:19.145068 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:19.144991 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-45bw2"] Jul 15 19:55:19.148045 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:19.148014 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb263d27_b8f9_4b8c_8297_a584a6742fdb.slice/crio-16247a0894abdc8aa08a93d9fa347bd58d1a416a505f5352642c43ae870de5bb WatchSource:0}: Error finding container 16247a0894abdc8aa08a93d9fa347bd58d1a416a505f5352642c43ae870de5bb: Status 404 returned error can't find the container with id 16247a0894abdc8aa08a93d9fa347bd58d1a416a505f5352642c43ae870de5bb Jul 15 19:55:19.201750 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:19.201721 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tr84l" event={"ID":"4e54a4d4-ffa7-4e74-8cad-960dceb020ab","Type":"ContainerStarted","Data":"346e8f635503a96b4b04f009380a80af56b43a4fc4d71473ce3884b8a7ab36d9"} Jul 15 19:55:19.202708 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:19.202680 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-45bw2" event={"ID":"bb263d27-b8f9-4b8c-8297-a584a6742fdb","Type":"ContainerStarted","Data":"16247a0894abdc8aa08a93d9fa347bd58d1a416a505f5352642c43ae870de5bb"} Jul 15 19:55:21.093151 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:21.093110 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:55:21.093745 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:21.093292 2557 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 15 19:55:21.093745 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:21.093313 2557 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7f77bc4dff-4k6n2: secret "image-registry-tls" not found Jul 15 19:55:21.093745 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:21.093386 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls podName:bb996b87-aef6-49fd-b396-1a91c740b141 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:53.093363532 +0000 UTC m=+100.764150038 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls") pod "image-registry-7f77bc4dff-4k6n2" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141") : secret "image-registry-tls" not found Jul 15 19:55:21.193879 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:21.193824 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:55:21.194073 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:21.194010 2557 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 19:55:21.194137 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:21.194075 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls podName:2602891b-0d4f-4354-964e-c028e469d84b nodeName:}" failed. No retries permitted until 2026-07-15 19:55:53.194059441 +0000 UTC m=+100.864845944 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls") pod "dns-default-f7wqb" (UID: "2602891b-0d4f-4354-964e-c028e469d84b") : secret "dns-default-metrics-tls" not found Jul 15 19:55:24.217586 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.217544 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tr84l" event={"ID":"4e54a4d4-ffa7-4e74-8cad-960dceb020ab","Type":"ContainerStarted","Data":"475c100b771f5e1583bbca69d2c38ae9dcfe707955352297abc54c5aba6742d0"} Jul 15 19:55:24.218945 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.218921 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-45bw2" event={"ID":"bb263d27-b8f9-4b8c-8297-a584a6742fdb","Type":"ContainerStarted","Data":"ad59523e96376473e107b9c18a8efeb3163468071711a60f7f20791c0db07fa9"} Jul 15 19:55:24.219074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.219057 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:55:24.235852 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.235750 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tr84l" podStartSLOduration=66.321787025 podStartE2EDuration="1m11.235734006s" totalImagesPullingTime="4.913946981s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:24.235723389 +0000 UTC m=+71.906509915" watchObservedRunningTime="2026-07-15 19:55:24.235734006 +0000 UTC m=+71.906520532" Jul 15 19:55:24.253591 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.253546 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-45bw2" podStartSLOduration=66.619478321 podStartE2EDuration="1m11.253537803s" totalImagesPullingTime="4.634059482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:24.253220393 +0000 UTC m=+71.924006915" watchObservedRunningTime="2026-07-15 19:55:24.253537803 +0000 UTC m=+71.924324305" Jul 15 19:55:24.291337 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.291304 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:24.391771 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:24.391734 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:26.725386 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:26.725351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:26.802995 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:26.802959 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:34.253530 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.253388 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5"] Jul 15 19:55:34.255438 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.255417 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.261482 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.261461 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 15 19:55:34.261633 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.261606 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 15 19:55:34.261864 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.261847 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 15 19:55:34.261907 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.261890 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 15 19:55:34.266982 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.266966 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hbwcw\"" Jul 15 19:55:34.277902 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.277880 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5"] Jul 15 19:55:34.365652 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.365624 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67"] Jul 15 19:55:34.367469 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.367455 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.373795 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.373778 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 19:55:34.384380 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.384357 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s4z9\" (UniqueName: \"kubernetes.io/projected/0b396c83-04c1-4ad3-b875-a981a23a74a5-kube-api-access-6s4z9\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.384466 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.384388 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b396c83-04c1-4ad3-b875-a981a23a74a5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.384466 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.384406 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b396c83-04c1-4ad3-b875-a981a23a74a5-config\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.401018 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.400998 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 19:55:34.402076 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.402062 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 19:55:34.402123 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.402100 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 19:55:34.402123 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.402113 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-kj5mw\"" Jul 15 19:55:34.424047 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.424023 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67"] Jul 15 19:55:34.456445 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.456418 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j"] Jul 15 19:55:34.458298 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.458279 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" Jul 15 19:55:34.464273 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.464257 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-t7nsp\"" Jul 15 19:55:34.479270 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.479231 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j"] Jul 15 19:55:34.485696 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.485675 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.485800 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.485769 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6s4z9\" (UniqueName: \"kubernetes.io/projected/0b396c83-04c1-4ad3-b875-a981a23a74a5-kube-api-access-6s4z9\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.485867 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.485805 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b396c83-04c1-4ad3-b875-a981a23a74a5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.485867 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.485829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b396c83-04c1-4ad3-b875-a981a23a74a5-config\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.485998 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.485978 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.486058 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.486040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l2bzx\" (UniqueName: \"kubernetes.io/projected/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-kube-api-access-l2bzx\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.486371 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.486353 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0b396c83-04c1-4ad3-b875-a981a23a74a5-config\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.488153 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.488135 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0b396c83-04c1-4ad3-b875-a981a23a74a5-serving-cert\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.531326 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.531264 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s4z9\" (UniqueName: \"kubernetes.io/projected/0b396c83-04c1-4ad3-b875-a981a23a74a5-kube-api-access-6s4z9\") pod \"service-ca-operator-7695fc6c8c-ct7j5\" (UID: \"0b396c83-04c1-4ad3-b875-a981a23a74a5\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.563631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.563598 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" Jul 15 19:55:34.586499 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.586466 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.586626 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.586505 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l2bzx\" (UniqueName: \"kubernetes.io/projected/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-kube-api-access-l2bzx\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.586626 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.586525 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vj57l\" (UniqueName: \"kubernetes.io/projected/5b973bfe-f234-4961-b0b0-2853cb8534b6-kube-api-access-vj57l\") pod \"network-check-source-869dcfcdcd-j4t2j\" (UID: \"5b973bfe-f234-4961-b0b0-2853cb8534b6\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" Jul 15 19:55:34.586626 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.586556 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.586787 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:34.586677 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:34.586787 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:34.586760 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls podName:6d0326db-fe6e-4e6f-b8bf-fb8303d78358 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:35.086714401 +0000 UTC m=+82.757500903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gqj67" (UID: "6d0326db-fe6e-4e6f-b8bf-fb8303d78358") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:34.587175 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.587156 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.640228 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.640188 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l2bzx\" (UniqueName: \"kubernetes.io/projected/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-kube-api-access-l2bzx\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:34.687157 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.687119 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vj57l\" (UniqueName: \"kubernetes.io/projected/5b973bfe-f234-4961-b0b0-2853cb8534b6-kube-api-access-vj57l\") pod \"network-check-source-869dcfcdcd-j4t2j\" (UID: \"5b973bfe-f234-4961-b0b0-2853cb8534b6\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" Jul 15 19:55:34.708067 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.708037 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vj57l\" (UniqueName: \"kubernetes.io/projected/5b973bfe-f234-4961-b0b0-2853cb8534b6-kube-api-access-vj57l\") pod \"network-check-source-869dcfcdcd-j4t2j\" (UID: \"5b973bfe-f234-4961-b0b0-2853cb8534b6\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" Jul 15 19:55:34.731005 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.730975 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5"] Jul 15 19:55:34.737387 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:34.737363 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b396c83_04c1_4ad3_b875_a981a23a74a5.slice/crio-2bc5896af40d4401accb59a21d53945ceba2a8058c2240f62be8e698a531ba61 WatchSource:0}: Error finding container 2bc5896af40d4401accb59a21d53945ceba2a8058c2240f62be8e698a531ba61: Status 404 returned error can't find the container with id 2bc5896af40d4401accb59a21d53945ceba2a8058c2240f62be8e698a531ba61 Jul 15 19:55:34.766747 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.766723 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" Jul 15 19:55:34.918061 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.918017 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j"] Jul 15 19:55:34.920707 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:34.920683 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b973bfe_f234_4961_b0b0_2853cb8534b6.slice/crio-3a27823b22ea0a2e233f786b744a411ca722aee56fdde9def3184ed7a0409800 WatchSource:0}: Error finding container 3a27823b22ea0a2e233f786b744a411ca722aee56fdde9def3184ed7a0409800: Status 404 returned error can't find the container with id 3a27823b22ea0a2e233f786b744a411ca722aee56fdde9def3184ed7a0409800 Jul 15 19:55:34.925004 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:34.924975 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:35.052642 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:35.052604 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:35.090737 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:35.090652 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:35.090887 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:35.090798 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:35.090887 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:35.090860 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls podName:6d0326db-fe6e-4e6f-b8bf-fb8303d78358 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:36.090845532 +0000 UTC m=+83.761632034 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gqj67" (UID: "6d0326db-fe6e-4e6f-b8bf-fb8303d78358") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:35.248555 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:35.248514 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" event={"ID":"5b973bfe-f234-4961-b0b0-2853cb8534b6","Type":"ContainerStarted","Data":"b9d25c5ef91e5a2c7cc5d640d11cf160e679ed44d467979337282e0d55dce117"} Jul 15 19:55:35.248736 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:35.248614 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" event={"ID":"5b973bfe-f234-4961-b0b0-2853cb8534b6","Type":"ContainerStarted","Data":"3a27823b22ea0a2e233f786b744a411ca722aee56fdde9def3184ed7a0409800"} Jul 15 19:55:35.249734 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:35.249695 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" event={"ID":"0b396c83-04c1-4ad3-b875-a981a23a74a5","Type":"ContainerStarted","Data":"2bc5896af40d4401accb59a21d53945ceba2a8058c2240f62be8e698a531ba61"} Jul 15 19:55:35.279308 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:35.279253 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-j4t2j" podStartSLOduration=1.279215609 podStartE2EDuration="1.279215609s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:35.279209676 +0000 UTC m=+82.949996203" watchObservedRunningTime="2026-07-15 19:55:35.279215609 +0000 UTC m=+82.950002137" Jul 15 19:55:36.097550 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:36.097508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:36.097742 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:36.097632 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:36.097742 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:36.097709 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls podName:6d0326db-fe6e-4e6f-b8bf-fb8303d78358 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:38.09768793 +0000 UTC m=+85.768474448 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gqj67" (UID: "6d0326db-fe6e-4e6f-b8bf-fb8303d78358") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:36.842547 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:36.842512 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:36.925383 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:36.925348 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:37.256509 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:37.256476 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" event={"ID":"0b396c83-04c1-4ad3-b875-a981a23a74a5","Type":"ContainerStarted","Data":"3caab4db4a93791654ddd01e66e2acea100ab9d5106469c892acf481af833c77"} Jul 15 19:55:37.292315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:37.292267 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" podStartSLOduration=1.289083851 podStartE2EDuration="3.292227062s" totalImagesPullingTime="2.003143211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:37.289660788 +0000 UTC m=+84.960447312" watchObservedRunningTime="2026-07-15 19:55:37.292227062 +0000 UTC m=+84.963013587" Jul 15 19:55:38.114347 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.114303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:38.114750 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:38.114462 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:38.114750 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:38.114530 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls podName:6d0326db-fe6e-4e6f-b8bf-fb8303d78358 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.114514221 +0000 UTC m=+89.785300724 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gqj67" (UID: "6d0326db-fe6e-4e6f-b8bf-fb8303d78358") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:38.545305 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.545275 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:38.861552 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.861465 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:38.910935 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.910904 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps"] Jul 15 19:55:38.913889 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.913873 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" Jul 15 19:55:38.917202 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.917183 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 19:55:38.917315 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.917232 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7f7sm\"" Jul 15 19:55:38.918405 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.918390 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 19:55:38.939246 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.939219 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps"] Jul 15 19:55:38.988020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:38.987989 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:39.020853 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.020823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w2qg\" (UniqueName: \"kubernetes.io/projected/69dbd07b-dbcd-427f-b2e8-ac27e5f67545-kube-api-access-7w2qg\") pod \"migrator-7f59b54f6c-b4jps\" (UID: \"69dbd07b-dbcd-427f-b2e8-ac27e5f67545\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" Jul 15 19:55:39.089097 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.089067 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:39.121412 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.121351 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w2qg\" (UniqueName: \"kubernetes.io/projected/69dbd07b-dbcd-427f-b2e8-ac27e5f67545-kube-api-access-7w2qg\") pod \"migrator-7f59b54f6c-b4jps\" (UID: \"69dbd07b-dbcd-427f-b2e8-ac27e5f67545\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" Jul 15 19:55:39.131389 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.131363 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w2qg\" (UniqueName: \"kubernetes.io/projected/69dbd07b-dbcd-427f-b2e8-ac27e5f67545-kube-api-access-7w2qg\") pod \"migrator-7f59b54f6c-b4jps\" (UID: \"69dbd07b-dbcd-427f-b2e8-ac27e5f67545\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" Jul 15 19:55:39.222731 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.222696 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" Jul 15 19:55:39.321290 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.321231 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:39.378516 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.378495 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps"] Jul 15 19:55:39.380897 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:39.380873 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69dbd07b_dbcd_427f_b2e8_ac27e5f67545.slice/crio-f134fc0f9d76c51738a5570844dde694c9b6b6f28edf1f97f9cae7153365c13a WatchSource:0}: Error finding container f134fc0f9d76c51738a5570844dde694c9b6b6f28edf1f97f9cae7153365c13a: Status 404 returned error can't find the container with id f134fc0f9d76c51738a5570844dde694c9b6b6f28edf1f97f9cae7153365c13a Jul 15 19:55:39.417509 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:39.417478 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:40.265073 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.265030 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" event={"ID":"69dbd07b-dbcd-427f-b2e8-ac27e5f67545","Type":"ContainerStarted","Data":"f134fc0f9d76c51738a5570844dde694c9b6b6f28edf1f97f9cae7153365c13a"} Jul 15 19:55:40.438720 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.438684 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5dklm"] Jul 15 19:55:40.442016 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.441992 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.445317 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.445295 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 19:55:40.445741 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.445721 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-cfjnb\"" Jul 15 19:55:40.445838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.445793 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 19:55:40.446263 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.446221 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 19:55:40.458530 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.458511 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 19:55:40.465084 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.465055 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5dklm"] Jul 15 19:55:40.506584 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.506502 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:40.530852 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.530780 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dcpc2\" (UniqueName: \"kubernetes.io/projected/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-api-access-dcpc2\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.530852 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.530814 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.530852 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.530842 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9c10eff-c27b-48a5-bb31-ede810a60d26-crio-socket\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.531077 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.530872 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.531077 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.530903 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c10eff-c27b-48a5-bb31-ede810a60d26-data-volume\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.618655 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.618619 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:40.631868 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.631839 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dcpc2\" (UniqueName: \"kubernetes.io/projected/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-api-access-dcpc2\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.632011 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.631883 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.632011 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:40.631971 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:55:40.632140 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.632012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9c10eff-c27b-48a5-bb31-ede810a60d26-crio-socket\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.632140 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:40.632029 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:41.132009942 +0000 UTC m=+88.802796448 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : configmap "kube-rbac-proxy" not found Jul 15 19:55:40.632140 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.632071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.632140 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.632074 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/f9c10eff-c27b-48a5-bb31-ede810a60d26-crio-socket\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.632140 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.632101 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c10eff-c27b-48a5-bb31-ede810a60d26-data-volume\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.632422 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:40.632196 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:55:40.632422 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:40.632283 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:41.132268935 +0000 UTC m=+88.803055438 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : secret "insights-runtime-extractor-tls" not found Jul 15 19:55:40.632543 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.632524 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/f9c10eff-c27b-48a5-bb31-ede810a60d26-data-volume\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.643540 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.643519 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dcpc2\" (UniqueName: \"kubernetes.io/projected/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-api-access-dcpc2\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:40.997942 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:40.997915 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:41.093769 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.093663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:41.135977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.135951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:41.136084 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.135997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:41.136084 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:41.136071 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:55:41.136153 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:41.136125 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:55:41.136153 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:41.136131 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.136116202 +0000 UTC m=+89.806902705 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : configmap "kube-rbac-proxy" not found Jul 15 19:55:41.136231 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:41.136161 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:42.136150857 +0000 UTC m=+89.806937360 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : secret "insights-runtime-extractor-tls" not found Jul 15 19:55:41.269974 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.269945 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" event={"ID":"69dbd07b-dbcd-427f-b2e8-ac27e5f67545","Type":"ContainerStarted","Data":"79a56db1bf1e7346a264928a0619bafb73219ccb2e50301ef607958f058e44e0"} Jul 15 19:55:41.270334 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.269994 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" event={"ID":"69dbd07b-dbcd-427f-b2e8-ac27e5f67545","Type":"ContainerStarted","Data":"934cb8c4e5f483c04cdb49cb51c8aa4e1c59fb0a0820855d806fc5b71e1c1cf4"} Jul 15 19:55:41.295897 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.295852 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-b4jps" podStartSLOduration=1.85128711 podStartE2EDuration="3.295836499s" totalImagesPullingTime="1.444549389s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:41.294584892 +0000 UTC m=+88.965371417" watchObservedRunningTime="2026-07-15 19:55:41.295836499 +0000 UTC m=+88.966623023" Jul 15 19:55:41.499206 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:41.499157 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:42.144628 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:42.144595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:42.144834 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:42.144654 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:42.144834 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:42.144758 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:42.144834 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:42.144769 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:55:42.144834 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:42.144816 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls podName:6d0326db-fe6e-4e6f-b8bf-fb8303d78358 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:50.144801763 +0000 UTC m=+97.815588268 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gqj67" (UID: "6d0326db-fe6e-4e6f-b8bf-fb8303d78358") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:42.144834 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:42.144832 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:42.145018 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:42.144875 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:44.144864505 +0000 UTC m=+91.815651008 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : secret "insights-runtime-extractor-tls" not found Jul 15 19:55:42.145018 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:42.144908 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:55:42.145018 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:42.144953 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:44.144941787 +0000 UTC m=+91.815728289 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : configmap "kube-rbac-proxy" not found Jul 15 19:55:42.259129 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:42.259092 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:43.035630 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:43.035602 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4gk5v_0303819b-d5e8-4964-94ed-2b90452a5036/dns-node-resolver/0.log" Jul 15 19:55:44.037771 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:44.037740 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5l4v9_c14a3660-2a88-490e-b481-6a414a642760/node-ca/0.log" Jul 15 19:55:44.159407 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:44.159379 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:44.159547 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:44.159493 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:44.159547 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:44.159525 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:55:44.159634 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:44.159563 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:55:44.159634 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:44.159622 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:48.159601953 +0000 UTC m=+95.830388470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : configmap "kube-rbac-proxy" not found Jul 15 19:55:44.159707 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:44.159637 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:48.159630506 +0000 UTC m=+95.830417009 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : secret "insights-runtime-extractor-tls" not found Jul 15 19:55:44.600734 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:44.600695 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:44.729526 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:44.729493 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:44.884346 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:44.884258 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:45.033778 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:45.033751 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-blcpq_1f70d4bd-946c-415a-b6c7-6546d7760288/serve-healthcheck-canary/0.log" Jul 15 19:55:45.451886 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:45.451858 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b4jps_69dbd07b-dbcd-427f-b2e8-ac27e5f67545/migrator/0.log" Jul 15 19:55:45.639405 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:45.639370 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b4jps_69dbd07b-dbcd-427f-b2e8-ac27e5f67545/graceful-termination/0.log" Jul 15 19:55:45.663229 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:45.663197 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:45.847919 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:45.847834 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:45.958865 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:45.958828 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:46.096485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:46.096453 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:46.468723 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:46.468686 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:46.933552 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:46.933526 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:47.325849 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:47.325762 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:47.664451 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:47.664374 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:47.878428 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:47.878397 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:48.193399 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.193363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:48.193580 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.193414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:48.193580 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:48.193483 2557 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 19:55:48.193580 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:48.193500 2557 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 19:55:48.193580 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:48.193560 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:56.193545707 +0000 UTC m=+103.864332210 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : secret "insights-runtime-extractor-tls" not found Jul 15 19:55:48.193580 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:48.193575 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm podName:f9c10eff-c27b-48a5-bb31-ede810a60d26 nodeName:}" failed. No retries permitted until 2026-07-15 19:55:56.193568143 +0000 UTC m=+103.864354646 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm") pod "insights-runtime-extractor-5dklm" (UID: "f9c10eff-c27b-48a5-bb31-ede810a60d26") : configmap "kube-rbac-proxy" not found Jul 15 19:55:48.348093 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.348056 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:48.445864 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.445780 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:48.513174 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.513136 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:48.820205 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.820117 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:48.919805 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:48.919776 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:49.288927 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:49.288895 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:49.672628 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:49.672539 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:50.067381 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:50.067347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:50.206894 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:50.206863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:55:50.207064 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:50.207018 2557 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:50.207105 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:55:50.207084 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls podName:6d0326db-fe6e-4e6f-b8bf-fb8303d78358 nodeName:}" failed. No retries permitted until 2026-07-15 19:56:06.207067115 +0000 UTC m=+113.877853617 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gqj67" (UID: "6d0326db-fe6e-4e6f-b8bf-fb8303d78358") : secret "cluster-monitoring-operator-tls" not found Jul 15 19:55:50.261202 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:50.261173 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:50.730446 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:50.730415 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:50.905061 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:50.905020 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:51.027357 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:51.027272 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:51.724376 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:51.724342 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:51.901366 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:51.901325 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:52.073357 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:52.073275 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:52.477456 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:52.477425 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:52.664982 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:52.664953 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:53.067727 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.067692 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:53.129977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.129945 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:55:53.132461 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.132439 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"image-registry-7f77bc4dff-4k6n2\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:55:53.156403 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.156375 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-pndwd\"" Jul 15 19:55:53.164379 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.164358 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:55:53.230836 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.230710 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:55:53.233491 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.233459 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2602891b-0d4f-4354-964e-c028e469d84b-metrics-tls\") pod \"dns-default-f7wqb\" (UID: \"2602891b-0d4f-4354-964e-c028e469d84b\") " pod="openshift-dns/dns-default-f7wqb" Jul 15 19:55:53.301561 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.301524 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7f77bc4dff-4k6n2"] Jul 15 19:55:53.306542 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:53.306518 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb996b87_aef6_49fd_b396_1a91c740b141.slice/crio-3609721ae95a0e074bd8169b615d8822eabf6a1ce793efcd5aa7c4a34faba157 WatchSource:0}: Error finding container 3609721ae95a0e074bd8169b615d8822eabf6a1ce793efcd5aa7c4a34faba157: Status 404 returned error can't find the container with id 3609721ae95a0e074bd8169b615d8822eabf6a1ce793efcd5aa7c4a34faba157 Jul 15 19:55:53.495802 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.495774 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-c5qnq\"" Jul 15 19:55:53.503525 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.503502 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-f7wqb" Jul 15 19:55:53.646499 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.646462 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-f7wqb"] Jul 15 19:55:53.652468 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:53.652440 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2602891b_0d4f_4354_964e_c028e469d84b.slice/crio-69e9af48469d1b2fe93dd58a603454dbe4fb581dd080de1db8453cd9ddb9ea9b WatchSource:0}: Error finding container 69e9af48469d1b2fe93dd58a603454dbe4fb581dd080de1db8453cd9ddb9ea9b: Status 404 returned error can't find the container with id 69e9af48469d1b2fe93dd58a603454dbe4fb581dd080de1db8453cd9ddb9ea9b Jul 15 19:55:53.715485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:53.715455 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:54.061863 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.061816 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:54.310019 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.309979 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" event={"ID":"bb996b87-aef6-49fd-b396-1a91c740b141","Type":"ContainerStarted","Data":"64371fd04c8857e3f8bbb20dcd4c0e17b953300985695500b6dcf150b529a557"} Jul 15 19:55:54.310524 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.310043 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" event={"ID":"bb996b87-aef6-49fd-b396-1a91c740b141","Type":"ContainerStarted","Data":"3609721ae95a0e074bd8169b615d8822eabf6a1ce793efcd5aa7c4a34faba157"} Jul 15 19:55:54.310524 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.310071 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:55:54.311503 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.311462 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f7wqb" event={"ID":"2602891b-0d4f-4354-964e-c028e469d84b","Type":"ContainerStarted","Data":"69e9af48469d1b2fe93dd58a603454dbe4fb581dd080de1db8453cd9ddb9ea9b"} Jul 15 19:55:54.337936 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.337843 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" podStartSLOduration=101.337828159 podStartE2EDuration="1m41.337828159s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:54.336299235 +0000 UTC m=+102.007085778" watchObservedRunningTime="2026-07-15 19:55:54.337828159 +0000 UTC m=+102.008614683" Jul 15 19:55:54.878093 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.878055 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:54.981487 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:54.981451 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:55.222980 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.222953 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-45bw2" Jul 15 19:55:55.313797 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.313771 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:55.316534 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.316498 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f7wqb" event={"ID":"2602891b-0d4f-4354-964e-c028e469d84b","Type":"ContainerStarted","Data":"22046eb1d7e39354a0c5ed522e2f48244944558a398e4a0ca87e9d89795619c4"} Jul 15 19:55:55.316666 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.316560 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-f7wqb" Jul 15 19:55:55.316666 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.316578 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-f7wqb" event={"ID":"2602891b-0d4f-4354-964e-c028e469d84b","Type":"ContainerStarted","Data":"5a44a590e0119c8c73403680c1f80dc6318c026f5b68477982137dd2fdc280ac"} Jul 15 19:55:55.409006 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.408911 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-f7wqb" podStartSLOduration=64.984574385 podStartE2EDuration="1m6.408890334s" totalImagesPullingTime="1.424315949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:55:55.407255558 +0000 UTC m=+103.078042074" watchObservedRunningTime="2026-07-15 19:55:55.408890334 +0000 UTC m=+103.079676859" Jul 15 19:55:55.418893 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:55.418861 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:56.257143 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:56.257098 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:56.257336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:56.257167 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:56.257687 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:56.257659 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/f9c10eff-c27b-48a5-bb31-ede810a60d26-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:56.259635 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:56.259617 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/f9c10eff-c27b-48a5-bb31-ede810a60d26-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5dklm\" (UID: \"f9c10eff-c27b-48a5-bb31-ede810a60d26\") " pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:56.353161 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:56.353130 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5dklm" Jul 15 19:55:56.520272 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:56.520202 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5dklm"] Jul 15 19:55:56.522360 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:55:56.522334 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9c10eff_c27b_48a5_bb31_ede810a60d26.slice/crio-6433ec0ef7c82c8feae86b62bb2a44aa693b7108e3b997150dff9602e30141db WatchSource:0}: Error finding container 6433ec0ef7c82c8feae86b62bb2a44aa693b7108e3b997150dff9602e30141db: Status 404 returned error can't find the container with id 6433ec0ef7c82c8feae86b62bb2a44aa693b7108e3b997150dff9602e30141db Jul 15 19:55:57.326290 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:57.326225 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dklm" event={"ID":"f9c10eff-c27b-48a5-bb31-ede810a60d26","Type":"ContainerStarted","Data":"6433ec0ef7c82c8feae86b62bb2a44aa693b7108e3b997150dff9602e30141db"} Jul 15 19:55:58.359505 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:58.359461 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:55:58.478357 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:55:58.478309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:02.349306 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:02.349262 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dklm" event={"ID":"f9c10eff-c27b-48a5-bb31-ede810a60d26","Type":"ContainerStarted","Data":"5381eb0d1cb2dcf0c4f5aae9a10f82db74ce1232018f26efee8b964511e3fe01"} Jul 15 19:56:04.801174 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:04.801125 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:05.028856 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:05.028823 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:05.179509 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:05.179475 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:05.287107 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:05.287049 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:05.321809 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:05.321776 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-f7wqb" Jul 15 19:56:06.244275 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:06.244161 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:56:06.246852 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:06.246828 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/6d0326db-fe6e-4e6f-b8bf-fb8303d78358-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gqj67\" (UID: \"6d0326db-fe6e-4e6f-b8bf-fb8303d78358\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:56:06.365384 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:06.365332 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dklm" event={"ID":"f9c10eff-c27b-48a5-bb31-ede810a60d26","Type":"ContainerStarted","Data":"42724fcd8ec4bf7a5a034925710e7a047d53bde5c632f0eb144fe24e7a70e1dd"} Jul 15 19:56:06.475718 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:06.475676 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" Jul 15 19:56:06.667096 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:06.666964 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67"] Jul 15 19:56:06.670001 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:06.669967 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d0326db_fe6e_4e6f_b8bf_fb8303d78358.slice/crio-466d07d09a4a5356490eda1cc1adb9e07c422d2f1b5eea8211322962fcb00a86 WatchSource:0}: Error finding container 466d07d09a4a5356490eda1cc1adb9e07c422d2f1b5eea8211322962fcb00a86: Status 404 returned error can't find the container with id 466d07d09a4a5356490eda1cc1adb9e07c422d2f1b5eea8211322962fcb00a86 Jul 15 19:56:07.370503 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:07.370462 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" event={"ID":"6d0326db-fe6e-4e6f-b8bf-fb8303d78358","Type":"ContainerStarted","Data":"466d07d09a4a5356490eda1cc1adb9e07c422d2f1b5eea8211322962fcb00a86"} Jul 15 19:56:10.383369 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.383276 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5dklm" event={"ID":"f9c10eff-c27b-48a5-bb31-ede810a60d26","Type":"ContainerStarted","Data":"2c7914cafbe48081eeb66eeb5eddc271b8a3d86aa55fbb4a8d0ed70a2863c4fe"} Jul 15 19:56:10.384690 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.384666 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" event={"ID":"6d0326db-fe6e-4e6f-b8bf-fb8303d78358","Type":"ContainerStarted","Data":"c66424580a9922c8bb13108f71db17f440c18e56a64a2b504dafe5949e3b8e2a"} Jul 15 19:56:10.439336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.439291 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5dklm" podStartSLOduration=17.132044108 podStartE2EDuration="30.439276732s" totalImagesPullingTime="13.307232624s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 19:55:40 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:10.438924511 +0000 UTC m=+118.109711035" watchObservedRunningTime="2026-07-15 19:56:10.439276732 +0000 UTC m=+118.110063257" Jul 15 19:56:10.463017 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.462992 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:10.481849 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.481810 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gqj67" podStartSLOduration=33.212826917 podStartE2EDuration="36.481795932s" totalImagesPullingTime="3.268969015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:55:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:10.481336434 +0000 UTC m=+118.152122960" watchObservedRunningTime="2026-07-15 19:56:10.481795932 +0000 UTC m=+118.152582457" Jul 15 19:56:10.552783 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.552750 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:10.668157 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.668078 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:10.711077 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.711047 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9"] Jul 15 19:56:10.714307 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.714291 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:10.718618 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.718600 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-dqqfq\"" Jul 15 19:56:10.718948 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.718932 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 15 19:56:10.744133 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.744103 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9"] Jul 15 19:56:10.783519 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.783482 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93a015ac-a73a-4564-b664-f44768cc7667-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-t29t9\" (UID: \"93a015ac-a73a-4564-b664-f44768cc7667\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:10.884149 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:10.884112 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93a015ac-a73a-4564-b664-f44768cc7667-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-t29t9\" (UID: \"93a015ac-a73a-4564-b664-f44768cc7667\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:10.884341 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:10.884272 2557 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 15 19:56:10.884388 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:10.884351 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/93a015ac-a73a-4564-b664-f44768cc7667-tls-certificates podName:93a015ac-a73a-4564-b664-f44768cc7667 nodeName:}" failed. No retries permitted until 2026-07-15 19:56:11.384332328 +0000 UTC m=+119.055118832 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/93a015ac-a73a-4564-b664-f44768cc7667-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-t29t9" (UID: "93a015ac-a73a-4564-b664-f44768cc7667") : secret "prometheus-operator-admission-webhook-tls" not found Jul 15 19:56:11.129485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:11.129449 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:11.387963 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:11.387881 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93a015ac-a73a-4564-b664-f44768cc7667-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-t29t9\" (UID: \"93a015ac-a73a-4564-b664-f44768cc7667\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:11.390475 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:11.390451 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/93a015ac-a73a-4564-b664-f44768cc7667-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-t29t9\" (UID: \"93a015ac-a73a-4564-b664-f44768cc7667\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:11.624399 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:11.624368 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:11.818793 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:11.818755 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9"] Jul 15 19:56:11.821874 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:11.821841 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93a015ac_a73a_4564_b664_f44768cc7667.slice/crio-ee14f4831f7f6d3d366f5cf94d8533f92dbbd05a1e664b21be7c5cc9db7decaf WatchSource:0}: Error finding container ee14f4831f7f6d3d366f5cf94d8533f92dbbd05a1e664b21be7c5cc9db7decaf: Status 404 returned error can't find the container with id ee14f4831f7f6d3d366f5cf94d8533f92dbbd05a1e664b21be7c5cc9db7decaf Jul 15 19:56:12.394551 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:12.394512 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" event={"ID":"93a015ac-a73a-4564-b664-f44768cc7667","Type":"ContainerStarted","Data":"ee14f4831f7f6d3d366f5cf94d8533f92dbbd05a1e664b21be7c5cc9db7decaf"} Jul 15 19:56:13.169105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.169015 2557 patch_prober.go:28] interesting pod/image-registry-7f77bc4dff-4k6n2 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 19:56:13.169105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.169071 2557 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" podUID="bb996b87-aef6-49fd-b396-1a91c740b141" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 19:56:13.398810 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.398777 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" event={"ID":"93a015ac-a73a-4564-b664-f44768cc7667","Type":"ContainerStarted","Data":"2b15fc9b9ee7609def95bc70857efe6503acbc0d4daca1922c02f1c0d7258660"} Jul 15 19:56:13.399290 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.398965 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:13.403786 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.403764 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" Jul 15 19:56:13.429138 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.429094 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-t29t9" podStartSLOduration=2.423403824 podStartE2EDuration="3.429081774s" totalImagesPullingTime="1.00567795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:13.427162745 +0000 UTC m=+121.097949271" watchObservedRunningTime="2026-07-15 19:56:13.429081774 +0000 UTC m=+121.099868298" Jul 15 19:56:13.744013 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.743937 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:13.831973 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.831932 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wcf9q"] Jul 15 19:56:13.835184 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.835162 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:13.845221 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.845202 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 15 19:56:13.848916 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.848892 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 15 19:56:13.849015 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.848923 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-wmcdt\"" Jul 15 19:56:13.849267 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.849229 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 19:56:13.859506 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.859485 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:13.862562 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.862543 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wcf9q"] Jul 15 19:56:13.908536 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.908503 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wwqz8\" (UniqueName: \"kubernetes.io/projected/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-kube-api-access-wwqz8\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:13.908674 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.908549 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:13.908674 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.908605 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:13.908674 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.908647 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:13.940044 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:13.940015 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:14.009634 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.009566 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wwqz8\" (UniqueName: \"kubernetes.io/projected/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-kube-api-access-wwqz8\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.009634 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.009627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.009785 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.009673 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.009785 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.009706 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.010320 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.010298 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-metrics-client-ca\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.012246 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.012213 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.012345 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.012223 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.018653 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.018623 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wwqz8\" (UniqueName: \"kubernetes.io/projected/7a973d49-e6c6-4b6b-a698-6a3011c7c5e5-kube-api-access-wwqz8\") pod \"prometheus-operator-64785b6675-wcf9q\" (UID: \"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5\") " pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.143774 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.143732 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" Jul 15 19:56:14.274017 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.273935 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-wcf9q"] Jul 15 19:56:14.276995 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:14.276961 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a973d49_e6c6_4b6b_a698_6a3011c7c5e5.slice/crio-eab5dc46cce6b7e1640e4694a41288c953416f59e62d3582071ead9b392e43b8 WatchSource:0}: Error finding container eab5dc46cce6b7e1640e4694a41288c953416f59e62d3582071ead9b392e43b8: Status 404 returned error can't find the container with id eab5dc46cce6b7e1640e4694a41288c953416f59e62d3582071ead9b392e43b8 Jul 15 19:56:14.403424 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.403389 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" event={"ID":"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5","Type":"ContainerStarted","Data":"eab5dc46cce6b7e1640e4694a41288c953416f59e62d3582071ead9b392e43b8"} Jul 15 19:56:14.632101 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.632012 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:14.715399 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:14.715362 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:15.322957 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:15.322920 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:56:15.350574 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:15.350538 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:15.480696 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:15.480662 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:16.418134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:16.418097 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" event={"ID":"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5","Type":"ContainerStarted","Data":"91e70f296c2d65b5ebc3709e5a28cd6c9de5292b0a532c1d82f0a3ec768244d1"} Jul 15 19:56:16.418324 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:16.418179 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" event={"ID":"7a973d49-e6c6-4b6b-a698-6a3011c7c5e5","Type":"ContainerStarted","Data":"53ea2d4c6df23b4aa666d4f8accf18ffee38f55ce1fbd2e8a64a0e95444ff884"} Jul 15 19:56:16.440928 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:16.440877 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-wcf9q" podStartSLOduration=2.17718488 podStartE2EDuration="3.440862228s" totalImagesPullingTime="1.263677348s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:16.440012797 +0000 UTC m=+124.110799321" watchObservedRunningTime="2026-07-15 19:56:16.440862228 +0000 UTC m=+124.111648754" Jul 15 19:56:16.470908 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:16.470882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:16.569983 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:16.569956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:18.096062 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.096026 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:18.254390 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.254356 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:18.645576 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.645531 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:18.791934 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.791886 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:18.828456 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.828425 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-47w89"] Jul 15 19:56:18.831773 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.831757 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.836929 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.836909 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 19:56:18.837071 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.836999 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-b5n5g\"" Jul 15 19:56:18.837128 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.837115 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 19:56:18.846791 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.846769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8wcgd\" (UniqueName: \"kubernetes.io/projected/ee492500-72c2-497a-8d75-856b40542110-kube-api-access-8wcgd\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.846887 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.846799 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee492500-72c2-497a-8d75-856b40542110-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.846887 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.846839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee492500-72c2-497a-8d75-856b40542110-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.846956 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.846914 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee492500-72c2-497a-8d75-856b40542110-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.913146 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.913071 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-47w89"] Jul 15 19:56:18.930357 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.930330 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:18.947531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.947506 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee492500-72c2-497a-8d75-856b40542110-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.947649 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.947539 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee492500-72c2-497a-8d75-856b40542110-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.947649 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.947582 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8wcgd\" (UniqueName: \"kubernetes.io/projected/ee492500-72c2-497a-8d75-856b40542110-kube-api-access-8wcgd\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.947649 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.947602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee492500-72c2-497a-8d75-856b40542110-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.948180 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.948155 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ee492500-72c2-497a-8d75-856b40542110-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.950184 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.950162 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ee492500-72c2-497a-8d75-856b40542110-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:18.950307 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:18.950215 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/ee492500-72c2-497a-8d75-856b40542110-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:19.037010 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.036981 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-drrqp"] Jul 15 19:56:19.040251 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.040220 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.050536 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.050513 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8wcgd\" (UniqueName: \"kubernetes.io/projected/ee492500-72c2-497a-8d75-856b40542110-kube-api-access-8wcgd\") pod \"openshift-state-metrics-84bd68b896-47w89\" (UID: \"ee492500-72c2-497a-8d75-856b40542110\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:19.057705 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.057688 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 15 19:56:19.084562 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.084536 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-zw4bx\"" Jul 15 19:56:19.084728 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.084702 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 15 19:56:19.084893 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.084875 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 15 19:56:19.091190 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.091171 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-drrqp"] Jul 15 19:56:19.140628 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.140599 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" Jul 15 19:56:19.148830 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.148801 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk5kp\" (UniqueName: \"kubernetes.io/projected/293af30d-2d20-4958-accf-5902f7005566-kube-api-access-kk5kp\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.148936 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.148849 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.148936 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.148883 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/293af30d-2d20-4958-accf-5902f7005566-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.148936 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.148915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.149078 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.149025 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/293af30d-2d20-4958-accf-5902f7005566-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.149114 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.149080 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.213348 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.213317 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qjvbj"] Jul 15 19:56:19.217427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.217404 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.228460 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:19.228436 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-tls\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" type="*v1.Secret" Jul 15 19:56:19.231000 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:19.230964 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"node-exporter-qjvbj\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" podUID="029d47f2-a823-4943-a139-6c7256afc076" pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.233196 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:19.233160 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"node-exporter-dockercfg-dpkjd\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dpkjd\"" type="*v1.Secret" Jul 15 19:56:19.233518 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.233487 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:19.249800 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.249768 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.249934 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.249828 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.249934 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.249876 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/293af30d-2d20-4958-accf-5902f7005566-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250014 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.249927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250014 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.249962 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-sys\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250014 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.249987 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87v7l\" (UniqueName: \"kubernetes.io/projected/029d47f2-a823-4943-a139-6c7256afc076-kube-api-access-87v7l\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250121 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk5kp\" (UniqueName: \"kubernetes.io/projected/293af30d-2d20-4958-accf-5902f7005566-kube-api-access-kk5kp\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250121 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250037 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250121 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250072 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-root\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250121 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-textfile\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250343 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250143 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/293af30d-2d20-4958-accf-5902f7005566-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250343 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250199 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-tls\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250343 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250252 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-wtmp\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250343 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250294 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250343 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250327 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029d47f2-a823-4943-a139-6c7256afc076-metrics-client-ca\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.250583 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250482 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/293af30d-2d20-4958-accf-5902f7005566-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.250705 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.250676 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/293af30d-2d20-4958-accf-5902f7005566-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.251170 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.251135 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.252657 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.252625 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.252760 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.252738 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/293af30d-2d20-4958-accf-5902f7005566-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.266017 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.265994 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 19:56:19.266109 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.265994 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 19:56:19.347132 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.347103 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk5kp\" (UniqueName: \"kubernetes.io/projected/293af30d-2d20-4958-accf-5902f7005566-kube-api-access-kk5kp\") pod \"kube-state-metrics-785778ff74-drrqp\" (UID: \"293af30d-2d20-4958-accf-5902f7005566\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.348878 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.348861 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" Jul 15 19:56:19.351170 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351150 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-root\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351256 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351178 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-textfile\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351256 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351199 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-tls\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351256 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351213 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-wtmp\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351270 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029d47f2-a823-4943-a139-6c7256afc076-metrics-client-ca\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351273 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-root\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351293 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351350 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351375 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-wtmp\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351386 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-sys\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351723 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351460 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/029d47f2-a823-4943-a139-6c7256afc076-sys\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351723 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351419 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87v7l\" (UniqueName: \"kubernetes.io/projected/029d47f2-a823-4943-a139-6c7256afc076-kube-api-access-87v7l\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.351723 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351567 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-textfile\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.352073 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.351915 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/029d47f2-a823-4943-a139-6c7256afc076-metrics-client-ca\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.352201 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.352184 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-accelerators-collector-config\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.353658 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.353637 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.360763 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:19.360742 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee492500_72c2_497a_8d75_856b40542110.slice/crio-3ad2f9c32bf52af9d26982fb510530080d419b8341d3027982c260f7540b2030 WatchSource:0}: Error finding container 3ad2f9c32bf52af9d26982fb510530080d419b8341d3027982c260f7540b2030: Status 404 returned error can't find the container with id 3ad2f9c32bf52af9d26982fb510530080d419b8341d3027982c260f7540b2030 Jul 15 19:56:19.363395 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.363379 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-47w89"] Jul 15 19:56:19.377136 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.377108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-87v7l\" (UniqueName: \"kubernetes.io/projected/029d47f2-a823-4943-a139-6c7256afc076-kube-api-access-87v7l\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:19.431569 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.431536 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" event={"ID":"ee492500-72c2-497a-8d75-856b40542110","Type":"ContainerStarted","Data":"3ad2f9c32bf52af9d26982fb510530080d419b8341d3027982c260f7540b2030"} Jul 15 19:56:19.518968 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.518936 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-drrqp"] Jul 15 19:56:19.520461 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:19.520429 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod293af30d_2d20_4958_accf_5902f7005566.slice/crio-1bc6ccc84b1ce75d74abd1d5bfa7ec1ac759375ea710e22faf0611d40f3d1f34 WatchSource:0}: Error finding container 1bc6ccc84b1ce75d74abd1d5bfa7ec1ac759375ea710e22faf0611d40f3d1f34: Status 404 returned error can't find the container with id 1bc6ccc84b1ce75d74abd1d5bfa7ec1ac759375ea710e22faf0611d40f3d1f34 Jul 15 19:56:19.856763 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:19.856683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:20.031971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.031888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:20.139066 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.138830 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 19:56:20.145612 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.145584 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/029d47f2-a823-4943-a139-6c7256afc076-node-exporter-tls\") pod \"node-exporter-qjvbj\" (UID: \"029d47f2-a823-4943-a139-6c7256afc076\") " pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:20.229622 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.229582 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:20.365538 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.365501 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:56:20.370020 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.369994 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.421900 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.421827 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 19:56:20.423571 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.423549 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 19:56:20.436899 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.436866 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" event={"ID":"293af30d-2d20-4958-accf-5902f7005566","Type":"ContainerStarted","Data":"1bc6ccc84b1ce75d74abd1d5bfa7ec1ac759375ea710e22faf0611d40f3d1f34"} Jul 15 19:56:20.438528 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.438502 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" event={"ID":"ee492500-72c2-497a-8d75-856b40542110","Type":"ContainerStarted","Data":"ab1b5edab99e63d87f62c1531c48c4491fcaa3613a4ff7cd6d745fda16d861c7"} Jul 15 19:56:20.438649 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.438556 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" event={"ID":"ee492500-72c2-497a-8d75-856b40542110","Type":"ContainerStarted","Data":"808424f8ea4633093d728ddd79bd3fbbd28209257a520d24319a41f09529a8d3"} Jul 15 19:56:20.445178 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.445148 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 19:56:20.445178 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.445172 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 19:56:20.445922 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.445181 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 19:56:20.445922 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.445303 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 19:56:20.445922 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.445624 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 19:56:20.446711 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.446692 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-dpkjd\"" Jul 15 19:56:20.448872 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.448856 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qjvbj" Jul 15 19:56:20.454127 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.454098 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-26zdj\"" Jul 15 19:56:20.462356 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462332 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462517 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462384 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462517 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462449 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462515 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kj2fp\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-kube-api-access-kj2fp\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462633 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462812 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462669 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462812 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462812 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-web-config\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.462812 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462800 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.463069 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462830 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.463069 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462871 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-config-volume\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.463069 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.462903 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-config-out\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.470649 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.470629 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 19:56:20.489732 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.489710 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 19:56:20.564182 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564147 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564380 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564196 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-config-volume\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564380 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564220 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-config-out\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564380 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564301 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564380 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564334 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564380 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564364 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564404 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kj2fp\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-kube-api-access-kj2fp\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564440 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564498 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564528 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564554 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-web-config\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.564715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.564567 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.565620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.565334 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.566356 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.566291 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.566813 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.566788 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.567884 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.567837 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.568886 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.568862 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-config-volume\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.569317 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.569275 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.569906 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.569863 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.569999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.569972 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.570554 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.570508 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-config-out\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.570706 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.570651 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-web-config\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.570951 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.570932 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.571487 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.571446 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-tls-assets\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.592359 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.592324 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:56:20.597993 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.597969 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kj2fp\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-kube-api-access-kj2fp\") pod \"alertmanager-main-0\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.599363 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.599335 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:20.682587 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:20.682558 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:56:20.840440 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:20.840401 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod029d47f2_a823_4943_a139_6c7256afc076.slice/crio-4f0c31176ce23e87246180fc66a783420e35abb369f9539e6368d74560c4f15c WatchSource:0}: Error finding container 4f0c31176ce23e87246180fc66a783420e35abb369f9539e6368d74560c4f15c: Status 404 returned error can't find the container with id 4f0c31176ce23e87246180fc66a783420e35abb369f9539e6368d74560c4f15c Jul 15 19:56:21.090207 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.090154 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:56:21.092310 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:21.092272 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840f91b0_22b8_4247_814a_dc98b554799e.slice/crio-62d1279e5bb362d0902187156a6836cb2bb5455ae0363a358c59fda7a07a267f WatchSource:0}: Error finding container 62d1279e5bb362d0902187156a6836cb2bb5455ae0363a358c59fda7a07a267f: Status 404 returned error can't find the container with id 62d1279e5bb362d0902187156a6836cb2bb5455ae0363a358c59fda7a07a267f Jul 15 19:56:21.455303 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.455252 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjvbj" event={"ID":"029d47f2-a823-4943-a139-6c7256afc076","Type":"ContainerStarted","Data":"4f0c31176ce23e87246180fc66a783420e35abb369f9539e6368d74560c4f15c"} Jul 15 19:56:21.457663 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.457634 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" event={"ID":"293af30d-2d20-4958-accf-5902f7005566","Type":"ContainerStarted","Data":"fb7cf5fb1c25edf615716461741e8af88e61a16b4032eab7c5e1f8a6d1e6bb2e"} Jul 15 19:56:21.457794 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.457693 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" event={"ID":"293af30d-2d20-4958-accf-5902f7005566","Type":"ContainerStarted","Data":"2bd05b6abf4ce60bf747e5691c2da04f256fdcf23e48b9ef8925a09e31a3cd40"} Jul 15 19:56:21.457794 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.457726 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" event={"ID":"293af30d-2d20-4958-accf-5902f7005566","Type":"ContainerStarted","Data":"5e6117fc8a402c57a4e66915f2511a078ad962ba1835753c0aec22d0697f1b56"} Jul 15 19:56:21.459340 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.459111 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"62d1279e5bb362d0902187156a6836cb2bb5455ae0363a358c59fda7a07a267f"} Jul 15 19:56:21.461496 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.461474 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" event={"ID":"ee492500-72c2-497a-8d75-856b40542110","Type":"ContainerStarted","Data":"c9c8c3fdf6fa344947362326dc4e3fab43cf67450f708c4929e3ec270f7300d8"} Jul 15 19:56:21.485895 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.485829 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-drrqp" podStartSLOduration=2.166712716 podStartE2EDuration="3.485812955s" totalImagesPullingTime="1.319100239s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:21.485582926 +0000 UTC m=+129.156369454" watchObservedRunningTime="2026-07-15 19:56:21.485812955 +0000 UTC m=+129.156599481" Jul 15 19:56:21.534474 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:21.534417 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-47w89" podStartSLOduration=2.203453662 podStartE2EDuration="3.534401219s" totalImagesPullingTime="1.330947557s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:21.529590128 +0000 UTC m=+129.200376652" watchObservedRunningTime="2026-07-15 19:56:21.534401219 +0000 UTC m=+129.205187744" Jul 15 19:56:22.161536 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.161492 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:22.257545 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.257479 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:22.394760 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.394670 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:22.586474 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.586426 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:56:22.589441 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.589410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed-metrics-certs\") pod \"network-metrics-daemon-ftj7m\" (UID: \"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed\") " pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:56:22.614474 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.614438 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-sjg99\"" Jul 15 19:56:22.621978 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.621956 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-ftj7m" Jul 15 19:56:22.885593 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.885491 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-ftj7m"] Jul 15 19:56:22.888222 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:22.888193 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e8df8a3_3ab7_444a_9ebd_c5c2041c74ed.slice/crio-9d7b139349372c977b50405b5c09a411f8c1c7b823236c9c9d41dea0c192c6f9 WatchSource:0}: Error finding container 9d7b139349372c977b50405b5c09a411f8c1c7b823236c9c9d41dea0c192c6f9: Status 404 returned error can't find the container with id 9d7b139349372c977b50405b5c09a411f8c1c7b823236c9c9d41dea0c192c6f9 Jul 15 19:56:22.891314 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.891286 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv"] Jul 15 19:56:22.895374 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.895309 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.898442 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.898422 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 15 19:56:22.898778 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.898730 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 15 19:56:22.899194 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.899178 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-fdlqc\"" Jul 15 19:56:22.899363 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.899347 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 15 19:56:22.899446 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.899352 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 15 19:56:22.899522 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.899504 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-eo7eeqgghsd3l\"" Jul 15 19:56:22.900728 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.900702 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 15 19:56:22.924414 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.924380 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv"] Jul 15 19:56:22.988728 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988686 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-tls\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.988728 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988728 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.988981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988757 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.988981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988783 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wzhcc\" (UniqueName: \"kubernetes.io/projected/86d1f514-a169-40f4-9745-243de995e52a-kube-api-access-wzhcc\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.988981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988836 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.988981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988882 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.988981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.988908 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86d1f514-a169-40f4-9745-243de995e52a-metrics-client-ca\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:22.989264 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:22.989040 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-grpc-tls\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090367 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090263 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-grpc-tls\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090367 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090319 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-tls\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090367 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090349 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090387 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090414 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wzhcc\" (UniqueName: \"kubernetes.io/projected/86d1f514-a169-40f4-9745-243de995e52a-kube-api-access-wzhcc\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090447 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090496 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.090636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.090521 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86d1f514-a169-40f4-9745-243de995e52a-metrics-client-ca\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.091973 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.091942 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/86d1f514-a169-40f4-9745-243de995e52a-metrics-client-ca\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.093580 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.093550 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.093685 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.093561 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.093739 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.093685 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.093739 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.093690 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.093821 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.093719 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-grpc-tls\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.093887 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.093869 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/86d1f514-a169-40f4-9745-243de995e52a-secret-thanos-querier-tls\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.123347 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.123319 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wzhcc\" (UniqueName: \"kubernetes.io/projected/86d1f514-a169-40f4-9745-243de995e52a-kube-api-access-wzhcc\") pod \"thanos-querier-6f7cf95d8f-62pxv\" (UID: \"86d1f514-a169-40f4-9745-243de995e52a\") " pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.128978 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.128952 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl"] Jul 15 19:56:23.133421 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.133400 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.142975 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.142958 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 19:56:23.143862 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.143847 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-fqq46\"" Jul 15 19:56:23.151361 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.151335 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl"] Jul 15 19:56:23.152436 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.152415 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:23.191302 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.191267 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjj48\" (UniqueName: \"kubernetes.io/projected/80159a3f-84dc-4a89-9973-c863520fb525-kube-api-access-cjj48\") pod \"monitoring-plugin-5f68df7f58-mbjjl\" (UID: \"80159a3f-84dc-4a89-9973-c863520fb525\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.191448 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.191317 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/80159a3f-84dc-4a89-9973-c863520fb525-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mbjjl\" (UID: \"80159a3f-84dc-4a89-9973-c863520fb525\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.251129 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.251096 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:23.292633 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.292595 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjj48\" (UniqueName: \"kubernetes.io/projected/80159a3f-84dc-4a89-9973-c863520fb525-kube-api-access-cjj48\") pod \"monitoring-plugin-5f68df7f58-mbjjl\" (UID: \"80159a3f-84dc-4a89-9973-c863520fb525\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.292793 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.292653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/80159a3f-84dc-4a89-9973-c863520fb525-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mbjjl\" (UID: \"80159a3f-84dc-4a89-9973-c863520fb525\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.295184 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.295165 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/80159a3f-84dc-4a89-9973-c863520fb525-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mbjjl\" (UID: \"80159a3f-84dc-4a89-9973-c863520fb525\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.318133 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.318104 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjj48\" (UniqueName: \"kubernetes.io/projected/80159a3f-84dc-4a89-9973-c863520fb525-kube-api-access-cjj48\") pod \"monitoring-plugin-5f68df7f58-mbjjl\" (UID: \"80159a3f-84dc-4a89-9973-c863520fb525\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.348339 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.348268 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:23.412314 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.412161 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv"] Jul 15 19:56:23.414767 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:23.414733 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d1f514_a169_40f4_9745_243de995e52a.slice/crio-88f05aa0736b7bbcd56095480bcd370fe97d237314420badf445337f5a447855 WatchSource:0}: Error finding container 88f05aa0736b7bbcd56095480bcd370fe97d237314420badf445337f5a447855: Status 404 returned error can't find the container with id 88f05aa0736b7bbcd56095480bcd370fe97d237314420badf445337f5a447855 Jul 15 19:56:23.439977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.439948 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:23.442880 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.442862 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:23.471411 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.471368 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"88f05aa0736b7bbcd56095480bcd370fe97d237314420badf445337f5a447855"} Jul 15 19:56:23.472869 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.472843 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ftj7m" event={"ID":"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed","Type":"ContainerStarted","Data":"9d7b139349372c977b50405b5c09a411f8c1c7b823236c9c9d41dea0c192c6f9"} Jul 15 19:56:23.474450 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.474423 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21" exitCode=0 Jul 15 19:56:23.474542 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.474501 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21"} Jul 15 19:56:23.476372 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.476301 2557 generic.go:356] "Generic (PLEG): container finished" podID="029d47f2-a823-4943-a139-6c7256afc076" containerID="b61fe4ccf3a2a26f1375419f7d9c7b0fc64deba0564f87cd4d8402101bc3b19c" exitCode=0 Jul 15 19:56:23.476372 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.476350 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjvbj" event={"ID":"029d47f2-a823-4943-a139-6c7256afc076","Type":"ContainerDied","Data":"b61fe4ccf3a2a26f1375419f7d9c7b0fc64deba0564f87cd4d8402101bc3b19c"} Jul 15 19:56:23.632735 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.632700 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl"] Jul 15 19:56:23.636427 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:23.636401 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80159a3f_84dc_4a89_9973_c863520fb525.slice/crio-cefe58ceb96eeb5f01d68a12fcb2cd1545cac161e5663666fc9bcc0446f80b79 WatchSource:0}: Error finding container cefe58ceb96eeb5f01d68a12fcb2cd1545cac161e5663666fc9bcc0446f80b79: Status 404 returned error can't find the container with id cefe58ceb96eeb5f01d68a12fcb2cd1545cac161e5663666fc9bcc0446f80b79 Jul 15 19:56:23.840340 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.840306 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-597f6b7d7d-bv76z"] Jul 15 19:56:23.843676 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.843659 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.846375 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.846346 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-client-certs\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" type="*v1.Secret" Jul 15 19:56:23.848458 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.848432 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-dockercfg-p6nk8\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-p6nk8\"" type="*v1.Secret" Jul 15 19:56:23.848535 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.848440 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-11bn9fq1agc6j\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-11bn9fq1agc6j\"" type="*v1.Secret" Jul 15 19:56:23.848843 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.848808 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"metrics-server-597f6b7d7d-bv76z\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" podUID="0f8f0728-7d76-4422-936e-03b6c92cabbb" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.848947 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.848922 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"metrics-server-audit-profiles\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" type="*v1.ConfigMap" Jul 15 19:56:23.849077 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.849053 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"metrics-server-tls\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" type="*v1.Secret" Jul 15 19:56:23.849077 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:23.849057 2557 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"kubelet-serving-ca-bundle\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-monitoring\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" type="*v1.ConfigMap" Jul 15 19:56:23.866366 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.866295 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-597f6b7d7d-bv76z"] Jul 15 19:56:23.898221 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898189 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-secret-metrics-server-client-certs\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.898221 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898224 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-client-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.898461 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-secret-metrics-server-tls\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.898461 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898336 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k94n5\" (UniqueName: \"kubernetes.io/projected/0f8f0728-7d76-4422-936e-03b6c92cabbb-kube-api-access-k94n5\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.898461 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898427 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f8f0728-7d76-4422-936e-03b6c92cabbb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.898461 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898456 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f8f0728-7d76-4422-936e-03b6c92cabbb-audit-log\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.898623 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.898501 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f8f0728-7d76-4422-936e-03b6c92cabbb-metrics-server-audit-profiles\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.999791 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.999751 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f8f0728-7d76-4422-936e-03b6c92cabbb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.999971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.999799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f8f0728-7d76-4422-936e-03b6c92cabbb-audit-log\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:23.999971 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.999928 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f8f0728-7d76-4422-936e-03b6c92cabbb-metrics-server-audit-profiles\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.000093 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:23.999984 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-secret-metrics-server-client-certs\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.000093 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.000021 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-client-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.000093 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.000071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-secret-metrics-server-tls\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.000320 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.000118 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k94n5\" (UniqueName: \"kubernetes.io/projected/0f8f0728-7d76-4422-936e-03b6c92cabbb-kube-api-access-k94n5\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.000320 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.000185 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/0f8f0728-7d76-4422-936e-03b6c92cabbb-audit-log\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.028272 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.028186 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k94n5\" (UniqueName: \"kubernetes.io/projected/0f8f0728-7d76-4422-936e-03b6c92cabbb-kube-api-access-k94n5\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.485909 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.485611 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjvbj" event={"ID":"029d47f2-a823-4943-a139-6c7256afc076","Type":"ContainerStarted","Data":"39c8083007a0708042b51ad7fabd522e5a4b6d459798fdebaef6d77d47fe6104"} Jul 15 19:56:24.485909 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.485680 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qjvbj" event={"ID":"029d47f2-a823-4943-a139-6c7256afc076","Type":"ContainerStarted","Data":"7fa01a2034f56eefc13e1f75cb0bb1e09ab0f037b7e6a9d209f5c20ab2686f66"} Jul 15 19:56:24.487300 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.487227 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" event={"ID":"80159a3f-84dc-4a89-9973-c863520fb525","Type":"ContainerStarted","Data":"cefe58ceb96eeb5f01d68a12fcb2cd1545cac161e5663666fc9bcc0446f80b79"} Jul 15 19:56:24.524410 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.524322 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qjvbj" podStartSLOduration=3.563234311 podStartE2EDuration="5.524299827s" totalImagesPullingTime="1.880242002s" totalInitContainerRuntime="80.823514ms" isStatefulPod=true podCreationTimestamp="2026-07-15 19:56:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:24.515910844 +0000 UTC m=+132.186697362" watchObservedRunningTime="2026-07-15 19:56:24.524299827 +0000 UTC m=+132.195086353" Jul 15 19:56:24.704584 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.704534 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 15 19:56:24.711372 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.711343 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0f8f0728-7d76-4422-936e-03b6c92cabbb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.723925 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.723894 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 15 19:56:24.732722 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.732688 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-secret-metrics-server-tls\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.907048 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.906960 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 15 19:56:24.915100 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.915068 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-secret-metrics-server-client-certs\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:24.968162 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.968129 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 15 19:56:24.971081 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:24.971057 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/0f8f0728-7d76-4422-936e-03b6c92cabbb-metrics-server-audit-profiles\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:25.000923 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:25.000854 2557 secret.go:189] Couldn't get secret openshift-monitoring/metrics-server-11bn9fq1agc6j: failed to sync secret cache: timed out waiting for the condition Jul 15 19:56:25.001095 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:56:25.000944 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-client-ca-bundle podName:0f8f0728-7d76-4422-936e-03b6c92cabbb nodeName:}" failed. No retries permitted until 2026-07-15 19:56:25.500920775 +0000 UTC m=+133.171707333 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "client-ca-bundle" (UniqueName: "kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-client-ca-bundle") pod "metrics-server-597f6b7d7d-bv76z" (UID: "0f8f0728-7d76-4422-936e-03b6c92cabbb") : failed to sync secret cache: timed out waiting for the condition Jul 15 19:56:25.321657 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.321621 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-11bn9fq1agc6j\"" Jul 15 19:56:25.325722 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.325699 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-p6nk8\"" Jul 15 19:56:25.379968 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.379322 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:25.504878 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.504850 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:25.513471 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.513105 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-client-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:25.515657 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.515632 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/0f8f0728-7d76-4422-936e-03b6c92cabbb-client-ca-bundle\") pod \"metrics-server-597f6b7d7d-bv76z\" (UID: \"0f8f0728-7d76-4422-936e-03b6c92cabbb\") " pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:25.654451 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.653946 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:25.659353 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.659324 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:25.673022 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.672997 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:56:25.677427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.677402 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.682554 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.682488 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 19:56:25.684186 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.684162 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 19:56:25.685066 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.685045 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 19:56:25.688777 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.688727 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 19:56:25.689039 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.689017 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 19:56:25.689119 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.688831 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 19:56:25.689871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.689852 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5jndp\"" Jul 15 19:56:25.690058 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.689852 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 19:56:25.690058 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.689854 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 19:56:25.694530 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.694510 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 19:56:25.695429 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.695412 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 19:56:25.695626 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.695589 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 19:56:25.695760 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.695741 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1nt79eph9v91r\"" Jul 15 19:56:25.695868 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.695797 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 19:56:25.697787 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.697766 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 19:56:25.715544 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715519 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715561 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715590 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715613 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715635 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715654 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715687 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config-out\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715739 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715767 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715789 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715803 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-web-config\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715864 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5bsqt\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-kube-api-access-5bsqt\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.715966 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715932 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.716532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.715999 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.716532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.716093 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.716532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.716168 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.716532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.716206 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.716532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.716256 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.723402 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.723373 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 19:56:25.723890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.723864 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:56:25.816879 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.816844 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.816896 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.816927 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.816944 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.816965 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.816996 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-web-config\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817023 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5bsqt\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-kube-api-access-5bsqt\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817066 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817162 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817222 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817303 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817329 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817365 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817404 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817420 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817445 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817485 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817482 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.817947 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.817524 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config-out\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.818949 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.818924 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.821108 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.821473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.821852 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.822208 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.822490 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.822488 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.822583 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.822987 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.822597 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-web-config\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.823798 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.823965 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.824444 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.825398 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.825448 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.825818 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.826206 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826568 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.826387 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config-out\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826619 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.826598 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.826692 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.826653 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.833022 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.832998 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5bsqt\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-kube-api-access-5bsqt\") pod \"prometheus-k8s-0\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:25.901874 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.901831 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:25.991692 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.991657 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:25.993810 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:25.993607 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:26.327745 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.327715 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-597f6b7d7d-bv76z"] Jul 15 19:56:26.339146 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:56:26.339099 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8f0728_7d76_4422_936e_03b6c92cabbb.slice/crio-431c4488bea03ae22178305662706ff32e817a89231cd39504a8cefede0244ab WatchSource:0}: Error finding container 431c4488bea03ae22178305662706ff32e817a89231cd39504a8cefede0244ab: Status 404 returned error can't find the container with id 431c4488bea03ae22178305662706ff32e817a89231cd39504a8cefede0244ab Jul 15 19:56:26.375072 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.375038 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:56:26.500043 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.499894 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ftj7m" event={"ID":"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed","Type":"ContainerStarted","Data":"2ea128c2e214643ba7041f78dc6dec46c0032a45c2e44d33cb64933e966cf81f"} Jul 15 19:56:26.500043 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.499964 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-ftj7m" event={"ID":"2e8df8a3-3ab7-444a-9ebd-c5c2041c74ed","Type":"ContainerStarted","Data":"a611d2b6b0121d7c2550cfc399d59805cea37354c6912c9730fa65a8078e838c"} Jul 15 19:56:26.505039 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.504916 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1"} Jul 15 19:56:26.505039 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.504975 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408"} Jul 15 19:56:26.505039 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.504999 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503"} Jul 15 19:56:26.511035 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.511007 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" event={"ID":"0f8f0728-7d76-4422-936e-03b6c92cabbb","Type":"ContainerStarted","Data":"431c4488bea03ae22178305662706ff32e817a89231cd39504a8cefede0244ab"} Jul 15 19:56:26.513462 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.513433 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"487672684e5d70b9c3106faf3628f76fd9244610768a503aacbeb64e997351c8"} Jul 15 19:56:26.513560 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.513512 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"cf1ad21ec59dfbdd72a2155bafd1751ab0520ab76eca87f216c67b1fbf86813e"} Jul 15 19:56:26.513560 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.513552 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"3444bbe9e3a91b771005ae3e67a188b9d2c6fa2143dd28cf014105ec5b43e364"} Jul 15 19:56:26.516327 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.515387 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed"} Jul 15 19:56:26.516327 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.515459 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"75957ce90eb46dfa0475054a83a2fd8bdedd4bc8daac267593e88d9685126e03"} Jul 15 19:56:26.518135 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.517976 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" event={"ID":"80159a3f-84dc-4a89-9973-c863520fb525","Type":"ContainerStarted","Data":"b30f34f52da2e5b61b36e2364e9bc1cbfb58a35a388ce96f4a77a2398aa41b90"} Jul 15 19:56:26.518383 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.518362 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:26.524963 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.524924 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" Jul 15 19:56:26.615375 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.615330 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-ftj7m" podStartSLOduration=131.363071012 podStartE2EDuration="2m14.615310745s" totalImagesPullingTime="3.252239733s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:54:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:26.558686033 +0000 UTC m=+134.229472559" watchObservedRunningTime="2026-07-15 19:56:26.615310745 +0000 UTC m=+134.286097270" Jul 15 19:56:26.701637 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.701580 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mbjjl" podStartSLOduration=1.191537806 podStartE2EDuration="3.70156352s" totalImagesPullingTime="2.510025714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:26.625789125 +0000 UTC m=+134.296575652" watchObservedRunningTime="2026-07-15 19:56:26.70156352 +0000 UTC m=+134.372350045" Jul 15 19:56:26.819759 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.819674 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:26.921497 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:26.921468 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:27.527953 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:27.527913 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a"} Jul 15 19:56:27.528130 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:27.528008 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846"} Jul 15 19:56:27.529964 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:27.529917 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed" exitCode=0 Jul 15 19:56:27.529964 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:27.529957 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed"} Jul 15 19:56:28.349306 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.349201 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:28.459789 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.459760 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:28.540202 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.540054 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerStarted","Data":"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f"} Jul 15 19:56:28.542104 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.542076 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" event={"ID":"0f8f0728-7d76-4422-936e-03b6c92cabbb","Type":"ContainerStarted","Data":"ddcf780269d1eb7dcbdc4e249956709b09077fdf2dc0cb9351efc41c0bf829d4"} Jul 15 19:56:28.545747 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.545718 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"96fbd6ab761839805d3ccafe8e5127aeff33903e7e0e313e8d97f74953ee4ee4"} Jul 15 19:56:28.545891 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.545805 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"cbf427b46844520a846149f2a4f71adbf041791e2feff84f0ff59d1a11ed0960"} Jul 15 19:56:28.545891 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.545850 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" event={"ID":"86d1f514-a169-40f4-9745-243de995e52a","Type":"ContainerStarted","Data":"7277daa178b987ec913dd7964241a62ec05c7391235bdc742976643af32ab11a"} Jul 15 19:56:28.583062 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.582975 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.03644427 podStartE2EDuration="8.582951212s" totalImagesPullingTime="5.526539853s" totalInitContainerRuntime="19.967089ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:28.580416613 +0000 UTC m=+136.251203157" watchObservedRunningTime="2026-07-15 19:56:28.582951212 +0000 UTC m=+136.253737737" Jul 15 19:56:28.614473 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.614365 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" podStartSLOduration=4.092998863 podStartE2EDuration="5.614341473s" totalImagesPullingTime="1.52134261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:28.613711417 +0000 UTC m=+136.284497961" watchObservedRunningTime="2026-07-15 19:56:28.614341473 +0000 UTC m=+136.285128000" Jul 15 19:56:28.653420 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:28.653373 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" podStartSLOduration=2.4903510779999998 podStartE2EDuration="6.653349039s" totalImagesPullingTime="4.162997961s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:28.651660703 +0000 UTC m=+136.322447241" watchObservedRunningTime="2026-07-15 19:56:28.653349039 +0000 UTC m=+136.324135570" Jul 15 19:56:29.476065 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:29.476023 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f77bc4dff-4k6n2"] Jul 15 19:56:29.551376 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:29.551348 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:29.641504 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:29.641466 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:29.755048 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:29.754960 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:30.362431 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.362366 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:30.539682 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.539566 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:30.559206 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.559176 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055"} Jul 15 19:56:30.559358 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.559334 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae"} Jul 15 19:56:30.559397 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.559369 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d"} Jul 15 19:56:30.559427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.559397 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a"} Jul 15 19:56:30.559470 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.559432 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c"} Jul 15 19:56:30.559470 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.559456 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerStarted","Data":"b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736"} Jul 15 19:56:30.637983 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.637929 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.107129736 podStartE2EDuration="5.637912834s" totalImagesPullingTime="3.508445758s" totalInitContainerRuntime="22.33734ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:56:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:56:30.62039417 +0000 UTC m=+138.291180713" watchObservedRunningTime="2026-07-15 19:56:30.637912834 +0000 UTC m=+138.308699359" Jul 15 19:56:30.930638 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.930602 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:30.994654 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:30.994617 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:56:31.120623 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:31.120587 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:31.281517 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:31.281428 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:32.310636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:32.310591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:32.459196 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:32.459162 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:35.565611 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:35.565585 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6f7cf95d8f-62pxv" Jul 15 19:56:35.646860 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:35.646830 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:35.746810 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:35.746773 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:37.260092 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:37.260047 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:37.353931 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:37.353888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:38.080005 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:38.079966 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:38.180890 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:38.180860 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:39.071205 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:39.071165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:39.152030 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:39.152002 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:43.309201 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:43.309167 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:43.376936 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:43.376904 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:45.309361 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:45.309324 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:45.407023 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:45.406991 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:45.654627 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:45.654543 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:45.654627 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:45.654602 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:56:49.475200 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:49.475166 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:49.545074 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:49.545039 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:54.498654 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.498585 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" podUID="bb996b87-aef6-49fd-b396-1a91c740b141" containerName="registry" containerID="cri-o://64371fd04c8857e3f8bbb20dcd4c0e17b953300985695500b6dcf150b529a557" gracePeriod=30 Jul 15 19:56:54.668150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.668115 2557 generic.go:356] "Generic (PLEG): container finished" podID="bb996b87-aef6-49fd-b396-1a91c740b141" containerID="64371fd04c8857e3f8bbb20dcd4c0e17b953300985695500b6dcf150b529a557" exitCode=0 Jul 15 19:56:54.668342 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.668185 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" event={"ID":"bb996b87-aef6-49fd-b396-1a91c740b141","Type":"ContainerDied","Data":"64371fd04c8857e3f8bbb20dcd4c0e17b953300985695500b6dcf150b529a557"} Jul 15 19:56:54.754532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.754479 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:56:54.793054 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793027 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2trl6\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-kube-api-access-2trl6\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793189 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793068 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-installation-pull-secrets\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793189 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793088 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-trusted-ca\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793189 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793110 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-bound-sa-token\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793354 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793251 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793354 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793290 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-image-registry-private-configuration\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793460 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793355 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-registry-certificates\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793460 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793446 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bb996b87-aef6-49fd-b396-1a91c740b141-ca-trust-extracted\") pod \"bb996b87-aef6-49fd-b396-1a91c740b141\" (UID: \"bb996b87-aef6-49fd-b396-1a91c740b141\") " Jul 15 19:56:54.793829 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793592 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:56:54.794037 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.793995 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:56:54.794194 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.794094 2557 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-registry-certificates\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.794194 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.794142 2557 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/bb996b87-aef6-49fd-b396-1a91c740b141-trusted-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.801294 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.796499 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:56:54.802309 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.801797 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:56:54.802710 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.802567 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:56:54.802891 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.802866 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-kube-api-access-2trl6" (OuterVolumeSpecName: "kube-api-access-2trl6") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "kube-api-access-2trl6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:56:54.802941 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.802877 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:56:54.806687 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.806655 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb996b87-aef6-49fd-b396-1a91c740b141-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "bb996b87-aef6-49fd-b396-1a91c740b141" (UID: "bb996b87-aef6-49fd-b396-1a91c740b141"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:56:54.895520 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.895490 2557 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/bb996b87-aef6-49fd-b396-1a91c740b141-ca-trust-extracted\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.895520 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.895516 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2trl6\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-kube-api-access-2trl6\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.895520 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.895525 2557 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-installation-pull-secrets\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.895722 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.895557 2557 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-bound-sa-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.895722 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.895566 2557 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/bb996b87-aef6-49fd-b396-1a91c740b141-registry-tls\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:54.895722 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:54.895575 2557 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/bb996b87-aef6-49fd-b396-1a91c740b141-image-registry-private-configuration\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:56:55.674025 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:55.674002 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" Jul 15 19:56:55.674432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:55.673994 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7f77bc4dff-4k6n2" event={"ID":"bb996b87-aef6-49fd-b396-1a91c740b141","Type":"ContainerDied","Data":"3609721ae95a0e074bd8169b615d8822eabf6a1ce793efcd5aa7c4a34faba157"} Jul 15 19:56:55.674432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:55.674170 2557 scope.go:117] "RemoveContainer" containerID="64371fd04c8857e3f8bbb20dcd4c0e17b953300985695500b6dcf150b529a557" Jul 15 19:56:55.696851 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:55.696818 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7f77bc4dff-4k6n2"] Jul 15 19:56:55.706387 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:55.706363 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7f77bc4dff-4k6n2"] Jul 15 19:56:56.784730 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:56.784696 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb996b87-aef6-49fd-b396-1a91c740b141" path="/var/lib/kubelet/pods/bb996b87-aef6-49fd-b396-1a91c740b141/volumes" Jul 15 19:56:57.302192 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:57.302155 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:58.368257 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:58.368207 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:58.445229 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:58.445194 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:58.917465 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:58.917427 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:56:59.504891 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:56:59.504844 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:00.280426 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:00.280395 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:00.378782 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:00.378754 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:00.678089 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:00.678059 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:00.740981 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:00.740943 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:01.536823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:01.536789 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:01.620795 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:01.620755 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:02.706397 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:02.706366 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b396c83-04c1-4ad3-b875-a981a23a74a5" containerID="3caab4db4a93791654ddd01e66e2acea100ab9d5106469c892acf481af833c77" exitCode=0 Jul 15 19:57:02.706799 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:02.706416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" event={"ID":"0b396c83-04c1-4ad3-b875-a981a23a74a5","Type":"ContainerDied","Data":"3caab4db4a93791654ddd01e66e2acea100ab9d5106469c892acf481af833c77"} Jul 15 19:57:02.706799 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:02.706764 2557 scope.go:117] "RemoveContainer" containerID="3caab4db4a93791654ddd01e66e2acea100ab9d5106469c892acf481af833c77" Jul 15 19:57:03.711822 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:03.711744 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-ct7j5" event={"ID":"0b396c83-04c1-4ad3-b875-a981a23a74a5","Type":"ContainerStarted","Data":"b67fc39d01b477bcdbad1b9c0e0df56cd4025cdc8ee33819a18adb45b639eb36"} Jul 15 19:57:05.660466 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:05.660436 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:57:05.664149 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:05.664123 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-597f6b7d7d-bv76z" Jul 15 19:57:10.649908 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:10.649874 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:10.722898 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:10.722866 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:13.319078 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:13.319042 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:13.384757 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:13.384709 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:14.438226 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:14.438194 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:20.856661 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:20.856628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:25.994295 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:25.994261 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:26.014999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:26.014975 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:26.831340 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:26.830968 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:38.495284 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:38.495223 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:38.799161 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:38.799071 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:39.597474 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597446 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:57:39.597872 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597773 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="alertmanager" containerID="cri-o://ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503" gracePeriod=120 Jul 15 19:57:39.597872 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597845 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-metric" containerID="cri-o://8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a" gracePeriod=120 Jul 15 19:57:39.597978 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597864 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="config-reloader" containerID="cri-o://226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408" gracePeriod=120 Jul 15 19:57:39.597978 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597922 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="prom-label-proxy" containerID="cri-o://90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f" gracePeriod=120 Jul 15 19:57:39.597978 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597930 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy" containerID="cri-o://da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846" gracePeriod=120 Jul 15 19:57:39.597978 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.597864 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-web" containerID="cri-o://9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1" gracePeriod=120 Jul 15 19:57:39.869631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869537 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f" exitCode=0 Jul 15 19:57:39.869631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869568 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846" exitCode=0 Jul 15 19:57:39.869631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869575 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408" exitCode=0 Jul 15 19:57:39.869631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869580 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503" exitCode=0 Jul 15 19:57:39.869631 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869609 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f"} Jul 15 19:57:39.869929 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869689 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846"} Jul 15 19:57:39.869929 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869727 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408"} Jul 15 19:57:39.869929 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:39.869765 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503"} Jul 15 19:57:40.851265 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.851224 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:40.882953 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.882916 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a" exitCode=0 Jul 15 19:57:40.882953 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.882947 2557 generic.go:356] "Generic (PLEG): container finished" podID="840f91b0-22b8-4247-814a-dc98b554799e" containerID="9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1" exitCode=0 Jul 15 19:57:40.883134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.883001 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:40.883134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.883002 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a"} Jul 15 19:57:40.883134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.883103 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1"} Jul 15 19:57:40.883134 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.883131 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"840f91b0-22b8-4247-814a-dc98b554799e","Type":"ContainerDied","Data":"62d1279e5bb362d0902187156a6836cb2bb5455ae0363a358c59fda7a07a267f"} Jul 15 19:57:40.883284 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.883177 2557 scope.go:117] "RemoveContainer" containerID="90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f" Jul 15 19:57:40.891002 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.890985 2557 scope.go:117] "RemoveContainer" containerID="8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a" Jul 15 19:57:40.900985 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.900963 2557 scope.go:117] "RemoveContainer" containerID="da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846" Jul 15 19:57:40.910107 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.910085 2557 scope.go:117] "RemoveContainer" containerID="9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1" Jul 15 19:57:40.919082 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919060 2557 scope.go:117] "RemoveContainer" containerID="226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408" Jul 15 19:57:40.919231 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919211 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-main-tls\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919364 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919287 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-cluster-tls-config\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919364 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919350 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-config-volume\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919477 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919379 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-config-out\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919477 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919417 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-web\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919477 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919447 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-trusted-ca-bundle\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919622 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919502 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-web-config\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919622 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919533 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-metric\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919622 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919580 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-metrics-client-ca\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919622 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919616 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-tls-assets\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919816 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919644 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919816 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919687 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kj2fp\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-kube-api-access-kj2fp\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.919816 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919733 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-main-db\") pod \"840f91b0-22b8-4247-814a-dc98b554799e\" (UID: \"840f91b0-22b8-4247-814a-dc98b554799e\") " Jul 15 19:57:40.920001 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.919973 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:40.920794 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.920293 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:40.922001 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.921965 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:57:40.924767 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.924710 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-config-volume" (OuterVolumeSpecName: "config-volume") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.925105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.924886 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.925105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.924915 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-config-out" (OuterVolumeSpecName: "config-out") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:57:40.925105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.924973 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.925105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.924994 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.925105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.925009 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.925105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.925020 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:57:40.926613 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.926590 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-kube-api-access-kj2fp" (OuterVolumeSpecName: "kube-api-access-kj2fp") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "kube-api-access-kj2fp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:57:40.930295 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.930274 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.937128 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.937109 2557 scope.go:117] "RemoveContainer" containerID="ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503" Jul 15 19:57:40.937379 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.937359 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-web-config" (OuterVolumeSpecName: "web-config") pod "840f91b0-22b8-4247-814a-dc98b554799e" (UID: "840f91b0-22b8-4247-814a-dc98b554799e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:40.945503 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.945485 2557 scope.go:117] "RemoveContainer" containerID="65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21" Jul 15 19:57:40.952732 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.952717 2557 scope.go:117] "RemoveContainer" containerID="90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f" Jul 15 19:57:40.953003 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.952983 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f\": container with ID starting with 90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f not found: ID does not exist" containerID="90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f" Jul 15 19:57:40.953068 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953010 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f"} err="failed to get container status \"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f\": rpc error: code = NotFound desc = could not find container \"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f\": container with ID starting with 90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f not found: ID does not exist" Jul 15 19:57:40.953068 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953041 2557 scope.go:117] "RemoveContainer" containerID="8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a" Jul 15 19:57:40.953300 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.953282 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a\": container with ID starting with 8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a not found: ID does not exist" containerID="8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a" Jul 15 19:57:40.953401 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953304 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a"} err="failed to get container status \"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a\": rpc error: code = NotFound desc = could not find container \"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a\": container with ID starting with 8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a not found: ID does not exist" Jul 15 19:57:40.953401 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953318 2557 scope.go:117] "RemoveContainer" containerID="da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846" Jul 15 19:57:40.953539 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.953524 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846\": container with ID starting with da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846 not found: ID does not exist" containerID="da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846" Jul 15 19:57:40.953577 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953543 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846"} err="failed to get container status \"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846\": rpc error: code = NotFound desc = could not find container \"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846\": container with ID starting with da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846 not found: ID does not exist" Jul 15 19:57:40.953577 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953555 2557 scope.go:117] "RemoveContainer" containerID="9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1" Jul 15 19:57:40.953796 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.953783 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1\": container with ID starting with 9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1 not found: ID does not exist" containerID="9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1" Jul 15 19:57:40.953837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953800 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1"} err="failed to get container status \"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1\": rpc error: code = NotFound desc = could not find container \"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1\": container with ID starting with 9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1 not found: ID does not exist" Jul 15 19:57:40.953837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.953811 2557 scope.go:117] "RemoveContainer" containerID="226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408" Jul 15 19:57:40.954058 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.954018 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408\": container with ID starting with 226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408 not found: ID does not exist" containerID="226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408" Jul 15 19:57:40.954099 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954061 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408"} err="failed to get container status \"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408\": rpc error: code = NotFound desc = could not find container \"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408\": container with ID starting with 226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408 not found: ID does not exist" Jul 15 19:57:40.954099 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954073 2557 scope.go:117] "RemoveContainer" containerID="ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503" Jul 15 19:57:40.954347 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.954330 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503\": container with ID starting with ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503 not found: ID does not exist" containerID="ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503" Jul 15 19:57:40.954403 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954354 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503"} err="failed to get container status \"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503\": rpc error: code = NotFound desc = could not find container \"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503\": container with ID starting with ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503 not found: ID does not exist" Jul 15 19:57:40.954403 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954371 2557 scope.go:117] "RemoveContainer" containerID="65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21" Jul 15 19:57:40.954615 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:40.954595 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21\": container with ID starting with 65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21 not found: ID does not exist" containerID="65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21" Jul 15 19:57:40.954679 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954625 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21"} err="failed to get container status \"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21\": rpc error: code = NotFound desc = could not find container \"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21\": container with ID starting with 65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21 not found: ID does not exist" Jul 15 19:57:40.954679 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954650 2557 scope.go:117] "RemoveContainer" containerID="90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f" Jul 15 19:57:40.954900 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954882 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f"} err="failed to get container status \"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f\": rpc error: code = NotFound desc = could not find container \"90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f\": container with ID starting with 90c0354fc5db64db25a82b1b4fbf101fc3fe0eb67518d9b4c47ed9efb87ac23f not found: ID does not exist" Jul 15 19:57:40.954948 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.954901 2557 scope.go:117] "RemoveContainer" containerID="8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a" Jul 15 19:57:40.955116 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955097 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a"} err="failed to get container status \"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a\": rpc error: code = NotFound desc = could not find container \"8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a\": container with ID starting with 8b09703f6db70f6bf10a3c60b4f6e3b34c6d9aef5162ba7d28c5e852ba8ffe8a not found: ID does not exist" Jul 15 19:57:40.955164 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955117 2557 scope.go:117] "RemoveContainer" containerID="da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846" Jul 15 19:57:40.955363 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955346 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846"} err="failed to get container status \"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846\": rpc error: code = NotFound desc = could not find container \"da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846\": container with ID starting with da0f6ffaf60bdc7174cf3986cb93e61b5951b0d932c6826ebf16793cce3da846 not found: ID does not exist" Jul 15 19:57:40.955433 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955364 2557 scope.go:117] "RemoveContainer" containerID="9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1" Jul 15 19:57:40.955613 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955596 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1"} err="failed to get container status \"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1\": rpc error: code = NotFound desc = could not find container \"9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1\": container with ID starting with 9d0dcef8669dfa05a2803cbafd7e86ecb6dd54a3f5bd569fff2c3a8ac8e620d1 not found: ID does not exist" Jul 15 19:57:40.955671 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955613 2557 scope.go:117] "RemoveContainer" containerID="226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408" Jul 15 19:57:40.955856 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955837 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408"} err="failed to get container status \"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408\": rpc error: code = NotFound desc = could not find container \"226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408\": container with ID starting with 226ad008ae112f9f1ce064b2de560085207a58076dc7f9e7e1be17d7f2275408 not found: ID does not exist" Jul 15 19:57:40.955932 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.955857 2557 scope.go:117] "RemoveContainer" containerID="ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503" Jul 15 19:57:40.956069 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.956051 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503"} err="failed to get container status \"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503\": rpc error: code = NotFound desc = could not find container \"ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503\": container with ID starting with ca7eb063053efb24a35064636c7ebe4b3001b681966fdb9adcdea755d5475503 not found: ID does not exist" Jul 15 19:57:40.956130 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.956070 2557 scope.go:117] "RemoveContainer" containerID="65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21" Jul 15 19:57:40.956358 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:40.956337 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21"} err="failed to get container status \"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21\": rpc error: code = NotFound desc = could not find container \"65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21\": container with ID starting with 65e3ab32c86cc496e4dcb201b5953ae71c830ec9f48f27c5adc182023f784d21 not found: ID does not exist" Jul 15 19:57:41.021351 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021321 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021351 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021344 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021355 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-web-config\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021386 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021399 2557 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/840f91b0-22b8-4247-814a-dc98b554799e-metrics-client-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021407 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-tls-assets\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021415 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021424 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kj2fp\" (UniqueName: \"kubernetes.io/projected/840f91b0-22b8-4247-814a-dc98b554799e-kube-api-access-kj2fp\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021432 2557 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-alertmanager-main-db\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021441 2557 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-secret-alertmanager-main-tls\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021449 2557 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-cluster-tls-config\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021460 2557 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/840f91b0-22b8-4247-814a-dc98b554799e-config-volume\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.021531 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.021472 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/840f91b0-22b8-4247-814a-dc98b554799e-config-out\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:41.221983 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.221891 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:57:41.228510 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.228485 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:57:41.273081 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273040 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:57:41.273467 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273448 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="alertmanager" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273468 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="alertmanager" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273489 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="init-config-reloader" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273496 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="init-config-reloader" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273514 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-web" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273523 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-web" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273532 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-metric" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273540 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-metric" Jul 15 19:57:41.273559 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273556 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="config-reloader" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273564 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="config-reloader" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273575 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="prom-label-proxy" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273584 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="prom-label-proxy" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273604 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb996b87-aef6-49fd-b396-1a91c740b141" containerName="registry" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273612 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb996b87-aef6-49fd-b396-1a91c740b141" containerName="registry" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273623 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273633 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273714 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="prom-label-proxy" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273726 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="alertmanager" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273739 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-metric" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273749 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy-web" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273760 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb996b87-aef6-49fd-b396-1a91c740b141" containerName="registry" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273771 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="config-reloader" Jul 15 19:57:41.273933 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.273780 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="840f91b0-22b8-4247-814a-dc98b554799e" containerName="kube-rbac-proxy" Jul 15 19:57:41.278088 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.278065 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.280993 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.280963 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 19:57:41.281389 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.281369 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 19:57:41.281513 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.281493 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 19:57:41.281651 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.281633 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 19:57:41.281772 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.281755 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 19:57:41.281861 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.281845 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 19:57:41.281970 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.281952 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 19:57:41.282142 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.282015 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 19:57:41.282330 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.282316 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-26zdj\"" Jul 15 19:57:41.288735 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.288718 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 19:57:41.295184 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.295162 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:57:41.324446 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324411 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a8ab20-b46f-4fb5-9fee-f932db077071-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324459 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-config-volume\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324484 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e0a8ab20-b46f-4fb5-9fee-f932db077071-config-out\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324543 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324566 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324594 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7w5m\" (UniqueName: \"kubernetes.io/projected/e0a8ab20-b46f-4fb5-9fee-f932db077071-kube-api-access-f7w5m\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324621 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324622 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-web-config\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324641 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a8ab20-b46f-4fb5-9fee-f932db077071-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324659 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a8ab20-b46f-4fb5-9fee-f932db077071-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324720 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324786 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.324838 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.324834 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a8ab20-b46f-4fb5-9fee-f932db077071-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.425930 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.425892 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a8ab20-b46f-4fb5-9fee-f932db077071-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.425930 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.425928 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a8ab20-b46f-4fb5-9fee-f932db077071-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.425954 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-config-volume\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.425973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e0a8ab20-b46f-4fb5-9fee-f932db077071-config-out\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426004 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426038 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426076 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7w5m\" (UniqueName: \"kubernetes.io/projected/e0a8ab20-b46f-4fb5-9fee-f932db077071-kube-api-access-f7w5m\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-web-config\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426133 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a8ab20-b46f-4fb5-9fee-f932db077071-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a8ab20-b46f-4fb5-9fee-f932db077071-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426187 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426219 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.426304 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426282 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.427053 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.426917 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e0a8ab20-b46f-4fb5-9fee-f932db077071-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.427111 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.427097 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0a8ab20-b46f-4fb5-9fee-f932db077071-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429447 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429263 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e0a8ab20-b46f-4fb5-9fee-f932db077071-config-out\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429447 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429349 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429447 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429390 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429447 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429428 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429474 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e0a8ab20-b46f-4fb5-9fee-f932db077071-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429531 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-config-volume\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429715 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429655 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e0a8ab20-b46f-4fb5-9fee-f932db077071-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.429868 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.429843 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.430355 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.430334 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-web-config\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.431250 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.431221 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e0a8ab20-b46f-4fb5-9fee-f932db077071-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.436825 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.436807 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7w5m\" (UniqueName: \"kubernetes.io/projected/e0a8ab20-b46f-4fb5-9fee-f932db077071-kube-api-access-f7w5m\") pod \"alertmanager-main-0\" (UID: \"e0a8ab20-b46f-4fb5-9fee-f932db077071\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.590296 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.590182 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 19:57:41.771902 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.771878 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 19:57:41.773625 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:57:41.773596 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0a8ab20_b46f_4fb5_9fee_f932db077071.slice/crio-86ad2f01cfed465131f88b053f03622d52c024ad4bdf3fc194f2eb6eefc8e490 WatchSource:0}: Error finding container 86ad2f01cfed465131f88b053f03622d52c024ad4bdf3fc194f2eb6eefc8e490: Status 404 returned error can't find the container with id 86ad2f01cfed465131f88b053f03622d52c024ad4bdf3fc194f2eb6eefc8e490 Jul 15 19:57:41.891977 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.891945 2557 generic.go:356] "Generic (PLEG): container finished" podID="e0a8ab20-b46f-4fb5-9fee-f932db077071" containerID="32c5378a6802b6bcd7c568b53cb7419d1a062536d2c36408494fd7801e66e138" exitCode=0 Jul 15 19:57:41.892441 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.892025 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerDied","Data":"32c5378a6802b6bcd7c568b53cb7419d1a062536d2c36408494fd7801e66e138"} Jul 15 19:57:41.892441 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:41.892120 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"86ad2f01cfed465131f88b053f03622d52c024ad4bdf3fc194f2eb6eefc8e490"} Jul 15 19:57:42.784166 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.784137 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="840f91b0-22b8-4247-814a-dc98b554799e" path="/var/lib/kubelet/pods/840f91b0-22b8-4247-814a-dc98b554799e/volumes" Jul 15 19:57:42.899562 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.899515 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"f04e53a990ef670d50908d4f790bfcaad737b49c6da864c1974aacbbe3b9a29c"} Jul 15 19:57:42.899999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.899629 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"d712705dcb717e80e1ccf77c3aa43c73f9e9cd73b77605f713abf8a55824d384"} Jul 15 19:57:42.899999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.899672 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"74c63c16e06c62da0a22312028e400355add0c71bf90ff8fc50043a4d5163491"} Jul 15 19:57:42.899999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.899708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"76d31754c8ea4ed962c35cbb8d871ce23112fdd4a508ccdcfd82ee3554d65f3e"} Jul 15 19:57:42.899999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.899741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"96c4f96984f6562cd6eab3ec08a93853a5c1ac640a3c8a3a7577e0ca48540b7e"} Jul 15 19:57:42.899999 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.899775 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e0a8ab20-b46f-4fb5-9fee-f932db077071","Type":"ContainerStarted","Data":"de1048794dc7c8c3c60d7a4a6ef173e508c5db7a0e58e0ee346bb230466403b1"} Jul 15 19:57:42.929366 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:42.929217 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=1.9132777669999999 podStartE2EDuration="1.929195434s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.917667ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:57:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:57:42.929022937 +0000 UTC m=+210.599809462" watchObservedRunningTime="2026-07-15 19:57:42.929195434 +0000 UTC m=+210.599981956" Jul 15 19:57:43.589686 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.589632 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-665cff56f5-p9tl4"] Jul 15 19:57:43.595173 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.595152 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.597887 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.597859 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 19:57:43.598065 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.597993 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-4z8s6\"" Jul 15 19:57:43.598065 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.597994 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 19:57:43.598206 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.597998 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 19:57:43.598206 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.598000 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 19:57:43.598383 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.598374 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 19:57:43.606645 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.606573 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 19:57:43.608880 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.608859 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-665cff56f5-p9tl4"] Jul 15 19:57:43.647532 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647499 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sfnl\" (UniqueName: \"kubernetes.io/projected/68085e92-7344-4419-b0f9-99ca5ebe54ee-kube-api-access-7sfnl\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647540 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647577 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-metrics-client-ca\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647612 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-federate-client-tls\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647634 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-serving-certs-ca-bundle\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647698 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647655 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-telemeter-client-tls\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647866 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647760 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-secret-telemeter-client\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.647866 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.647788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.748990 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.748948 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749161 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-metrics-client-ca\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749161 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749040 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-federate-client-tls\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749161 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749072 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-serving-certs-ca-bundle\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749161 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749103 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-telemeter-client-tls\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749455 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749171 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-secret-telemeter-client\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749455 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749195 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.749455 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.749273 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sfnl\" (UniqueName: \"kubernetes.io/projected/68085e92-7344-4419-b0f9-99ca5ebe54ee-kube-api-access-7sfnl\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.750544 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.750514 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-telemeter-trusted-ca-bundle\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.750913 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.750887 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-serving-certs-ca-bundle\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.751129 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.751107 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/68085e92-7344-4419-b0f9-99ca5ebe54ee-metrics-client-ca\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.753121 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.753093 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-secret-telemeter-client\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.753343 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.753321 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.753624 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.753603 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-telemeter-client-tls\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.753669 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.753604 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/68085e92-7344-4419-b0f9-99ca5ebe54ee-federate-client-tls\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.758799 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.758775 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sfnl\" (UniqueName: \"kubernetes.io/projected/68085e92-7344-4419-b0f9-99ca5ebe54ee-kube-api-access-7sfnl\") pod \"telemeter-client-665cff56f5-p9tl4\" (UID: \"68085e92-7344-4419-b0f9-99ca5ebe54ee\") " pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:43.900702 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.900602 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:57:43.901168 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.900990 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="prometheus" containerID="cri-o://b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736" gracePeriod=600 Jul 15 19:57:43.901168 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.901141 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy" containerID="cri-o://b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae" gracePeriod=600 Jul 15 19:57:43.901319 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.901180 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="thanos-sidecar" containerID="cri-o://de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a" gracePeriod=600 Jul 15 19:57:43.901319 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.901149 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-web" containerID="cri-o://e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d" gracePeriod=600 Jul 15 19:57:43.901319 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.901300 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="config-reloader" containerID="cri-o://af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c" gracePeriod=600 Jul 15 19:57:43.901602 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.901580 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055" gracePeriod=600 Jul 15 19:57:43.909117 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:43.906802 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" Jul 15 19:57:44.080106 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.080008 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-665cff56f5-p9tl4"] Jul 15 19:57:44.082359 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:57:44.082334 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68085e92_7344_4419_b0f9_99ca5ebe54ee.slice/crio-a52322ea4605667496b12c73b3a85ea739d698027e2e846c142291a324cfd288 WatchSource:0}: Error finding container a52322ea4605667496b12c73b3a85ea739d698027e2e846c142291a324cfd288: Status 404 returned error can't find the container with id a52322ea4605667496b12c73b3a85ea739d698027e2e846c142291a324cfd288 Jul 15 19:57:44.918336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918295 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055" exitCode=0 Jul 15 19:57:44.918336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918331 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae" exitCode=0 Jul 15 19:57:44.918336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918341 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a" exitCode=0 Jul 15 19:57:44.918336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918349 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c" exitCode=0 Jul 15 19:57:44.918336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918339 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055"} Jul 15 19:57:44.918336 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918357 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736" exitCode=0 Jul 15 19:57:44.919011 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918408 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae"} Jul 15 19:57:44.919011 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918436 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a"} Jul 15 19:57:44.919011 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918460 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c"} Jul 15 19:57:44.919011 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.918493 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736"} Jul 15 19:57:44.919775 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:44.919741 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" event={"ID":"68085e92-7344-4419-b0f9-99ca5ebe54ee","Type":"ContainerStarted","Data":"a52322ea4605667496b12c73b3a85ea739d698027e2e846c142291a324cfd288"} Jul 15 19:57:45.156205 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.156174 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:45.263674 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.263639 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-db\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.263837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.263685 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-tls-assets\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.263837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.263713 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-metrics-client-certs\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.263837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.263743 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.263837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.263781 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.263837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.263811 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5bsqt\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-kube-api-access-5bsqt\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264099 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264025 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-1\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264155 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264099 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264155 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264127 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-0\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264272 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264156 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-2\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264272 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264182 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-web-config\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264272 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264217 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-grpc-tls\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264269 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-kubelet-serving-ca-bundle\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264295 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-metrics-client-ca\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264336 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-kube-rbac-proxy\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264374 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-tls\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264427 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264399 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.264664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264430 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-trusted-ca-bundle\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264453 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config-out\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264502 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-serving-certs-ca-bundle\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264664 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264530 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-thanos-prometheus-http-client-file\") pod \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\" (UID: \"0b53a38e-68b3-4c25-b253-12abaa43bf9d\") " Jul 15 19:57:45.264907 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264801 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:57:45.265000 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264929 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.265000 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.264955 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.265605 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.265575 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.265605 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.265592 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.266535 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.266273 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.266535 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.266458 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.266704 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.266660 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 19:57:45.269756 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.269270 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-kube-api-access-5bsqt" (OuterVolumeSpecName: "kube-api-access-5bsqt") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "kube-api-access-5bsqt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:57:45.269756 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.269279 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 19:57:45.269756 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.269404 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.269756 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.269428 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config" (OuterVolumeSpecName: "config") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.270010 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.269841 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config-out" (OuterVolumeSpecName: "config-out") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 19:57:45.270193 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.270068 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.270193 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.270157 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.270437 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.270403 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.270526 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.270466 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.270570 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.270517 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.271086 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.271051 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.282398 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.282376 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-web-config" (OuterVolumeSpecName: "web-config") pod "0b53a38e-68b3-4c25-b253-12abaa43bf9d" (UID: "0b53a38e-68b3-4c25-b253-12abaa43bf9d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 19:57:45.365624 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365590 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365624 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365618 2557 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-thanos-prometheus-http-client-file\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365624 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365629 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-db\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365639 2557 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-tls-assets\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365647 2557 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-metrics-client-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365656 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365665 2557 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365673 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5bsqt\" (UniqueName: \"kubernetes.io/projected/0b53a38e-68b3-4c25-b253-12abaa43bf9d-kube-api-access-5bsqt\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365682 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365691 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365699 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365708 2557 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-web-config\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365716 2557 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-grpc-tls\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365724 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365732 2557 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-configmap-metrics-client-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365740 2557 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-kube-rbac-proxy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365748 2557 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/0b53a38e-68b3-4c25-b253-12abaa43bf9d-secret-prometheus-k8s-tls\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365757 2557 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b53a38e-68b3-4c25-b253-12abaa43bf9d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.365823 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.365765 2557 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/0b53a38e-68b3-4c25-b253-12abaa43bf9d-config-out\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 19:57:45.928342 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.928299 2557 generic.go:356] "Generic (PLEG): container finished" podID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerID="e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d" exitCode=0 Jul 15 19:57:45.928798 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.928361 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d"} Jul 15 19:57:45.928798 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.928401 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:45.928798 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.928447 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"0b53a38e-68b3-4c25-b253-12abaa43bf9d","Type":"ContainerDied","Data":"75957ce90eb46dfa0475054a83a2fd8bdedd4bc8daac267593e88d9685126e03"} Jul 15 19:57:45.928798 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.928482 2557 scope.go:117] "RemoveContainer" containerID="952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055" Jul 15 19:57:45.955375 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.955349 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:57:45.960650 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.960621 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:57:45.988871 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.988838 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989287 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-thanos" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989309 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-thanos" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989325 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="thanos-sidecar" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989334 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="thanos-sidecar" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989364 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="init-config-reloader" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989375 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="init-config-reloader" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989385 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="prometheus" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989393 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="prometheus" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989400 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="config-reloader" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989405 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="config-reloader" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989411 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-web" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989420 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-web" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989430 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989438 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989525 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="thanos-sidecar" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989540 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="config-reloader" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989550 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-web" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989560 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy-thanos" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989568 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="prometheus" Jul 15 19:57:45.991432 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.989575 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" containerName="kube-rbac-proxy" Jul 15 19:57:45.994927 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.994903 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:45.997934 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.997905 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 15 19:57:45.999221 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.999202 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 15 19:57:45.999340 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.999270 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-1nt79eph9v91r\"" Jul 15 19:57:45.999340 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:45.999270 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 15 19:57:46.001083 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.001041 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 15 19:57:46.001503 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.001484 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-5jndp\"" Jul 15 19:57:46.001842 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.001798 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 15 19:57:46.001910 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.001859 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 15 19:57:46.002090 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.002071 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 15 19:57:46.002275 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.002261 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 15 19:57:46.002450 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.002436 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 15 19:57:46.002636 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.002610 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 15 19:57:46.002894 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.002880 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 15 19:57:46.005758 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.005572 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 15 19:57:46.005908 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.005886 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 15 19:57:46.008389 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.008365 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 15 19:57:46.011120 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.011062 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:57:46.073743 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073707 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.073918 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1668ee16-4826-4fa6-a35a-2641ebc17919-config-out\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.073918 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073786 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.073918 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073846 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.073918 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073885 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.073918 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073964 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.073997 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-config\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074017 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1668ee16-4826-4fa6-a35a-2641ebc17919-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074038 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074075 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074117 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4lxp\" (UniqueName: \"kubernetes.io/projected/1668ee16-4826-4fa6-a35a-2641ebc17919-kube-api-access-p4lxp\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074141 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074141 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074253 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074292 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074322 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074361 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074393 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074425 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.074498 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.074473 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-web-config\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.143742 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.143595 2557 scope.go:117] "RemoveContainer" containerID="b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae" Jul 15 19:57:46.151470 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.151450 2557 scope.go:117] "RemoveContainer" containerID="e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d" Jul 15 19:57:46.158526 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.158508 2557 scope.go:117] "RemoveContainer" containerID="de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a" Jul 15 19:57:46.175821 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175800 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.175921 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.175921 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175848 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.175921 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175872 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.175921 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175893 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.175921 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175921 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175941 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-web-config\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175960 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.175994 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1668ee16-4826-4fa6-a35a-2641ebc17919-config-out\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176023 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176077 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176104 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176126 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176150 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176151 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-config\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176174 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1668ee16-4826-4fa6-a35a-2641ebc17919-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176255 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176306 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4lxp\" (UniqueName: \"kubernetes.io/projected/1668ee16-4826-4fa6-a35a-2641ebc17919-kube-api-access-p4lxp\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176333 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176447 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176620 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176975 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176634 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.176975 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.176855 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.179308 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.178204 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.179308 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.178818 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.179308 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.178992 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.179726 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.179698 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.180366 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.180124 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.180366 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.180295 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.181169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.181086 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1668ee16-4826-4fa6-a35a-2641ebc17919-config-out\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.181169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.181119 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.181169 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.181121 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1668ee16-4826-4fa6-a35a-2641ebc17919-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.181704 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.181662 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.182041 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.182010 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-web-config\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.182112 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.182067 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.182167 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.182153 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-config\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.183060 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.183042 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1668ee16-4826-4fa6-a35a-2641ebc17919-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.183104 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.183048 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1668ee16-4826-4fa6-a35a-2641ebc17919-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.190367 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.190348 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4lxp\" (UniqueName: \"kubernetes.io/projected/1668ee16-4826-4fa6-a35a-2641ebc17919-kube-api-access-p4lxp\") pod \"prometheus-k8s-0\" (UID: \"1668ee16-4826-4fa6-a35a-2641ebc17919\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.196681 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.196659 2557 scope.go:117] "RemoveContainer" containerID="af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c" Jul 15 19:57:46.208639 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.208624 2557 scope.go:117] "RemoveContainer" containerID="b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736" Jul 15 19:57:46.216207 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.216185 2557 scope.go:117] "RemoveContainer" containerID="d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed" Jul 15 19:57:46.223972 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.223951 2557 scope.go:117] "RemoveContainer" containerID="952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055" Jul 15 19:57:46.224252 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.224214 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055\": container with ID starting with 952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055 not found: ID does not exist" containerID="952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055" Jul 15 19:57:46.224318 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.224260 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055"} err="failed to get container status \"952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055\": rpc error: code = NotFound desc = could not find container \"952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055\": container with ID starting with 952ece37abe07ec723f1624d553e3351403ebe03fc6a63bb36ddae116480e055 not found: ID does not exist" Jul 15 19:57:46.224318 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.224280 2557 scope.go:117] "RemoveContainer" containerID="b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae" Jul 15 19:57:46.224561 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.224543 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae\": container with ID starting with b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae not found: ID does not exist" containerID="b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae" Jul 15 19:57:46.224641 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.224570 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae"} err="failed to get container status \"b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae\": rpc error: code = NotFound desc = could not find container \"b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae\": container with ID starting with b3769db1ddbfe93566cbed697953ecf7186867f79c5d718e623af08c0f4b2cae not found: ID does not exist" Jul 15 19:57:46.224641 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.224593 2557 scope.go:117] "RemoveContainer" containerID="e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d" Jul 15 19:57:46.224837 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.224818 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d\": container with ID starting with e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d not found: ID does not exist" containerID="e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d" Jul 15 19:57:46.224893 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.224841 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d"} err="failed to get container status \"e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d\": rpc error: code = NotFound desc = could not find container \"e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d\": container with ID starting with e224965e3907f6e026018f91447d10d547d08d982dc1a2225c2412ccf380095d not found: ID does not exist" Jul 15 19:57:46.224893 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.224853 2557 scope.go:117] "RemoveContainer" containerID="de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a" Jul 15 19:57:46.225069 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.225054 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a\": container with ID starting with de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a not found: ID does not exist" containerID="de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a" Jul 15 19:57:46.225111 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225074 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a"} err="failed to get container status \"de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a\": rpc error: code = NotFound desc = could not find container \"de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a\": container with ID starting with de637626a8186cf95c8686410e555e68249f33733f9105286a2f37507ad0557a not found: ID does not exist" Jul 15 19:57:46.225111 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225088 2557 scope.go:117] "RemoveContainer" containerID="af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c" Jul 15 19:57:46.225335 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.225321 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c\": container with ID starting with af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c not found: ID does not exist" containerID="af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c" Jul 15 19:57:46.225385 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225337 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c"} err="failed to get container status \"af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c\": rpc error: code = NotFound desc = could not find container \"af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c\": container with ID starting with af83bf3a61a957a2f0878662392a06d659a3ec97c2d16e085f7892421175479c not found: ID does not exist" Jul 15 19:57:46.225385 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225349 2557 scope.go:117] "RemoveContainer" containerID="b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736" Jul 15 19:57:46.225588 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.225566 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736\": container with ID starting with b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736 not found: ID does not exist" containerID="b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736" Jul 15 19:57:46.225642 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225594 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736"} err="failed to get container status \"b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736\": rpc error: code = NotFound desc = could not find container \"b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736\": container with ID starting with b3831b5f6929384c49e2019eadb1c6e1e5b314cbb34ff4314b113703350ae736 not found: ID does not exist" Jul 15 19:57:46.225642 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225619 2557 scope.go:117] "RemoveContainer" containerID="d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed" Jul 15 19:57:46.225896 ip-10-0-132-12 kubenswrapper[2557]: E0715 19:57:46.225879 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed\": container with ID starting with d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed not found: ID does not exist" containerID="d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed" Jul 15 19:57:46.225967 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.225896 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed"} err="failed to get container status \"d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed\": rpc error: code = NotFound desc = could not find container \"d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed\": container with ID starting with d3117e7bb7a2e9bdcb7040e96d9534fb9ad5cf456c8bf42855993087000f3aed not found: ID does not exist" Jul 15 19:57:46.307223 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.307159 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:46.483541 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.483516 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 15 19:57:46.485412 ip-10-0-132-12 kubenswrapper[2557]: W0715 19:57:46.485390 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1668ee16_4826_4fa6_a35a_2641ebc17919.slice/crio-dd6229194b1447d9bb6ba9ff30bcdda71ee01f7b6a3e4686bb990b29c16278b7 WatchSource:0}: Error finding container dd6229194b1447d9bb6ba9ff30bcdda71ee01f7b6a3e4686bb990b29c16278b7: Status 404 returned error can't find the container with id dd6229194b1447d9bb6ba9ff30bcdda71ee01f7b6a3e4686bb990b29c16278b7 Jul 15 19:57:46.784679 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.784603 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b53a38e-68b3-4c25-b253-12abaa43bf9d" path="/var/lib/kubelet/pods/0b53a38e-68b3-4c25-b253-12abaa43bf9d/volumes" Jul 15 19:57:46.934846 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.934805 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" event={"ID":"68085e92-7344-4419-b0f9-99ca5ebe54ee","Type":"ContainerStarted","Data":"8d1f99469ff1e81d3b3be56a07b2a09a065d664b06c6611b4aee8f89a17c6ee0"} Jul 15 19:57:46.935368 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.934871 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" event={"ID":"68085e92-7344-4419-b0f9-99ca5ebe54ee","Type":"ContainerStarted","Data":"3aa7f3c462ca1686f61d3a0759a6c84c36e7c535c6f163a7181ba47fa650b37a"} Jul 15 19:57:46.935368 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.934894 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" event={"ID":"68085e92-7344-4419-b0f9-99ca5ebe54ee","Type":"ContainerStarted","Data":"7325ee0ead03d036bdd8db9cdbf17c129212ebf7030057fc0a3d7a5ec436d7d6"} Jul 15 19:57:46.937113 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.937088 2557 generic.go:356] "Generic (PLEG): container finished" podID="1668ee16-4826-4fa6-a35a-2641ebc17919" containerID="7caf5c5b2ff67b77063ecc6cc4b273f7d913106a9498c749cac9caf322e90d36" exitCode=0 Jul 15 19:57:46.937231 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.937122 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerDied","Data":"7caf5c5b2ff67b77063ecc6cc4b273f7d913106a9498c749cac9caf322e90d36"} Jul 15 19:57:46.937231 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.937165 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"dd6229194b1447d9bb6ba9ff30bcdda71ee01f7b6a3e4686bb990b29c16278b7"} Jul 15 19:57:46.964925 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:46.964538 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-665cff56f5-p9tl4" podStartSLOduration=1.844803133 podStartE2EDuration="3.964518949s" totalImagesPullingTime="2.119715816s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 19:57:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:57:46.96415316 +0000 UTC m=+214.634939697" watchObservedRunningTime="2026-07-15 19:57:46.964518949 +0000 UTC m=+214.635305480" Jul 15 19:57:47.812808 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.812768 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:47.904800 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.904749 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:47.945895 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.945861 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"d1e897732fae2c1f8c35338e0ffa133168f8074e29151d6e90c91b6010d436bc"} Jul 15 19:57:47.946301 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.945941 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"48b1f1bff57814a50aa49961b54b762b8800d7d2af603ae3dd526e9b59a1daf9"} Jul 15 19:57:47.946301 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.945968 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"f37d8a057db879187ae1a834e005bd76d1e604b7ef512fd7013818c905fa36bc"} Jul 15 19:57:47.946301 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.946006 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"56c582e95986a27937f874cf25e9b5d11e497bb47ff349c77cc1ee904f18a7cc"} Jul 15 19:57:47.946301 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.946033 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"d5acf7e686d72ccd658c1dface493ff8cc21fa02608e32b6ff947bd90e808ab1"} Jul 15 19:57:47.946301 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.946057 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1668ee16-4826-4fa6-a35a-2641ebc17919","Type":"ContainerStarted","Data":"51adf589cd8e1940c8a1e3e0e46850dbdfd1fc198be36ccd5ad9ae6e3fd5bee9"} Jul 15 19:57:47.984108 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:47.984043 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.965375096 podStartE2EDuration="2.984020251s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.645155ms" isStatefulPod=false podCreationTimestamp="2026-07-15 19:57:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 19:57:47.981062452 +0000 UTC m=+215.651848977" watchObservedRunningTime="2026-07-15 19:57:47.984020251 +0000 UTC m=+215.654806777" Jul 15 19:57:51.307972 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:51.307937 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:57:59.019052 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:59.019016 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:57:59.141518 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:57:59.141480 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:44.547472 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:44.547434 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:44.647505 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:44.647471 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:46.308275 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:46.308218 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:58:46.324824 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:46.324791 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:58:47.212473 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:47.212443 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 15 19:58:53.750837 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:53.750803 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:53.848824 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:53.848779 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:57.929911 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:57.929875 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:58.026968 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:58.026934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:58.119486 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:58.119456 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:58.617635 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:58.617604 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:58:59.107383 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:58:59.107347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:05.803822 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:05.803786 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:05.903727 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:05.903668 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:09.253155 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:09.253117 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:09.348105 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:09.348066 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:12.705508 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:12.705481 2557 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 19:59:13.910436 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:13.910395 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:14.011946 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:14.011916 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:18.322795 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:18.322759 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:20.557922 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:20.557888 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:32.057470 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:32.057426 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:55.803972 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:55.803931 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 19:59:55.896651 ip-10-0-132-12 kubenswrapper[2557]: I0715 19:59:55.896617 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:19.412584 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:19.412485 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:22.003440 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:22.003406 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:23.180486 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:23.180452 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:23.285192 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:23.285152 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:24.711276 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:24.711218 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:24.818682 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:24.818631 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:27.710599 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.710567 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k"] Jul 15 20:00:27.713892 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.713874 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:27.717100 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.717075 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 20:00:27.717263 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.717161 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-rczmv\"" Jul 15 20:00:27.718047 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.718031 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 20:00:27.724060 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.724036 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k"] Jul 15 20:00:27.791535 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.791494 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zwr2\" (UniqueName: \"kubernetes.io/projected/0c6ee8e6-353c-4331-b9e1-1d66eb6ba015-kube-api-access-6zwr2\") pod \"cert-manager-cainjector-559cdd8c87-6gk2k\" (UID: \"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:27.791756 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.791545 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c6ee8e6-353c-4331-b9e1-1d66eb6ba015-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6gk2k\" (UID: \"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:27.892611 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.892574 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zwr2\" (UniqueName: \"kubernetes.io/projected/0c6ee8e6-353c-4331-b9e1-1d66eb6ba015-kube-api-access-6zwr2\") pod \"cert-manager-cainjector-559cdd8c87-6gk2k\" (UID: \"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:27.892824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.892627 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c6ee8e6-353c-4331-b9e1-1d66eb6ba015-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6gk2k\" (UID: \"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:27.901730 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.901699 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c6ee8e6-353c-4331-b9e1-1d66eb6ba015-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-6gk2k\" (UID: \"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:27.901854 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:27.901796 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zwr2\" (UniqueName: \"kubernetes.io/projected/0c6ee8e6-353c-4331-b9e1-1d66eb6ba015-kube-api-access-6zwr2\") pod \"cert-manager-cainjector-559cdd8c87-6gk2k\" (UID: \"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:28.038176 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:28.038083 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" Jul 15 20:00:28.176169 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:28.176140 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k"] Jul 15 20:00:28.178663 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:00:28.178637 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c6ee8e6_353c_4331_b9e1_1d66eb6ba015.slice/crio-e0163586b8f44a020c3f2accec7b89e35cbae91a48b5e2996f8dfc3f57e589cb WatchSource:0}: Error finding container e0163586b8f44a020c3f2accec7b89e35cbae91a48b5e2996f8dfc3f57e589cb: Status 404 returned error can't find the container with id e0163586b8f44a020c3f2accec7b89e35cbae91a48b5e2996f8dfc3f57e589cb Jul 15 20:00:28.180666 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:28.180648 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:00:28.311043 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:28.310959 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:28.416402 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:28.416369 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:28.619020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:28.618931 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" event={"ID":"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015","Type":"ContainerStarted","Data":"e0163586b8f44a020c3f2accec7b89e35cbae91a48b5e2996f8dfc3f57e589cb"} Jul 15 20:00:30.175836 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:30.175744 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:30.275740 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:30.275685 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:31.636307 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:31.636189 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" event={"ID":"0c6ee8e6-353c-4331-b9e1-1d66eb6ba015","Type":"ContainerStarted","Data":"4afa6564626bd8beea295cf1b26148b8c9bf92ff3b050b0eff15483fbc42383f"} Jul 15 20:00:31.657490 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:31.657436 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-6gk2k" podStartSLOduration=1.575772892 podStartE2EDuration="4.65741719s" totalImagesPullingTime="3.081644298s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:00:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:00:31.655585767 +0000 UTC m=+379.326372292" watchObservedRunningTime="2026-07-15 20:00:31.65741719 +0000 UTC m=+379.328203715" Jul 15 20:00:34.962722 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:34.962695 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:40.376283 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:40.376247 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:40.475393 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:40.475358 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:47.578001 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:47.577961 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:47.687518 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:47.687483 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:00:48.112140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:00:48.112103 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:01.305905 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:01.305873 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:01.407011 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:01.406981 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:05.310388 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.310356 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:05.386263 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.386202 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:05.651630 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.651548 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:05.752177 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.752146 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:05.864921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.864891 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc"] Jul 15 20:01:05.868864 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.868848 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:05.871683 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.871661 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-kg598\"" Jul 15 20:01:05.871921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.871905 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 20:01:05.872111 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.872097 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 20:01:05.872186 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.872099 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 20:01:05.872575 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.872562 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 20:01:05.887174 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.887144 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc"] Jul 15 20:01:05.923761 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.923677 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-manifests\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:05.923761 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.923753 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:05.924005 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.923833 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-tmp\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:05.924005 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.923868 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:05.924005 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:05.923905 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frbzd\" (UniqueName: \"kubernetes.io/projected/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-kube-api-access-frbzd\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.024698 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.024662 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.024889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.024710 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-tmp\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.024889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.024829 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.024889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.024877 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frbzd\" (UniqueName: \"kubernetes.io/projected/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-kube-api-access-frbzd\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.025059 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.025010 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-manifests\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.025468 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.025411 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-manifests\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.027450 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.027425 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-tmp\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.027563 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.027549 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-apiservice-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.027725 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.027694 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-webhook-cert\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.041961 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.041933 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frbzd\" (UniqueName: \"kubernetes.io/projected/27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae-kube-api-access-frbzd\") pod \"opendatahub-operator-controller-manager-57c4dc5856-8cwmc\" (UID: \"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae\") " pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.110611 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.110569 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.179453 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.179428 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:06.204921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.204887 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.301566 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.301533 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.333707 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.333685 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc"] Jul 15 20:01:06.335966 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:01:06.335940 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27a2a7fc_a7ea_4d1d_b92c_b5fa87f2a6ae.slice/crio-df62d00db7b6dfa62850e0b102910e23ee2302953707a30ec3322ab1c72ff96e WatchSource:0}: Error finding container df62d00db7b6dfa62850e0b102910e23ee2302953707a30ec3322ab1c72ff96e: Status 404 returned error can't find the container with id df62d00db7b6dfa62850e0b102910e23ee2302953707a30ec3322ab1c72ff96e Jul 15 20:01:06.375769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.375739 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.475928 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.475846 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.575855 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.575820 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.679193 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.679162 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:06.785879 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:06.785791 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" event={"ID":"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae","Type":"ContainerStarted","Data":"df62d00db7b6dfa62850e0b102910e23ee2302953707a30ec3322ab1c72ff96e"} Jul 15 20:01:10.805908 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:10.805862 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" event={"ID":"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae","Type":"ContainerStarted","Data":"7451d5a1b0fa0507586beff6de6036e3aa86925661b43bbd84a5bd5a2ab57587"} Jul 15 20:01:11.811449 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:11.811411 2557 generic.go:356] "Generic (PLEG): container finished" podID="27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae" containerID="7451d5a1b0fa0507586beff6de6036e3aa86925661b43bbd84a5bd5a2ab57587" exitCode=0 Jul 15 20:01:11.811894 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:11.811502 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" event={"ID":"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae","Type":"ContainerDied","Data":"7451d5a1b0fa0507586beff6de6036e3aa86925661b43bbd84a5bd5a2ab57587"} Jul 15 20:01:12.818664 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:12.818624 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" event={"ID":"27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae","Type":"ContainerStarted","Data":"557ab2b13d55519141755240ad6528c2cce0659180f6230cb1e74f75a8235c11"} Jul 15 20:01:12.819129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:12.818726 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:12.867018 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:12.866956 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" podStartSLOduration=3.053162842 podStartE2EDuration="7.866934481s" totalImagesPullingTime="3.735971518s" totalInitContainerRuntime="1.077800121s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:12.866318795 +0000 UTC m=+420.537105320" watchObservedRunningTime="2026-07-15 20:01:12.866934481 +0000 UTC m=+420.537721007" Jul 15 20:01:14.369940 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:14.369903 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:14.463285 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:14.463211 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:23.186370 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:23.186337 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:23.309615 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:23.309581 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:23.604051 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:23.603965 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:23.694719 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:23.694678 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:23.825916 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:23.825886 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-57c4dc5856-8cwmc" Jul 15 20:01:27.448306 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:27.448271 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:27.539741 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:27.539696 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:27.878595 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:27.878519 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:28.196472 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:28.196440 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:28.401143 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:28.401105 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:28.515832 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:28.515747 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:28.633116 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:28.633073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:28.769794 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:28.769693 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:29.522351 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:29.522317 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:29.758972 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:29.758939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:29.978578 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:29.978534 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:30.191373 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:30.191342 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:32.189019 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:32.188985 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:36.488583 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.488508 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9"] Jul 15 20:01:36.491962 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.491945 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.495127 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.495102 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 15 20:01:36.496086 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.496070 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 15 20:01:36.496180 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.496115 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 15 20:01:36.496180 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.496124 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 15 20:01:36.496416 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.496396 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 15 20:01:36.496464 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.496447 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-z48ch\"" Jul 15 20:01:36.504367 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.504347 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5bed4249-443e-4f02-8ba2-22e57169577e-manager-config\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.504453 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.504377 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5bed4249-443e-4f02-8ba2-22e57169577e-metrics-cert\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.504453 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.504407 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bed4249-443e-4f02-8ba2-22e57169577e-cert\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.504534 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.504503 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhqnq\" (UniqueName: \"kubernetes.io/projected/5bed4249-443e-4f02-8ba2-22e57169577e-kube-api-access-lhqnq\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.505889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.505868 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9"] Jul 15 20:01:36.605058 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.605022 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5bed4249-443e-4f02-8ba2-22e57169577e-metrics-cert\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.605227 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.605071 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bed4249-443e-4f02-8ba2-22e57169577e-cert\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.605227 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.605108 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lhqnq\" (UniqueName: \"kubernetes.io/projected/5bed4249-443e-4f02-8ba2-22e57169577e-kube-api-access-lhqnq\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.605227 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.605165 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5bed4249-443e-4f02-8ba2-22e57169577e-manager-config\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.605882 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.605852 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/5bed4249-443e-4f02-8ba2-22e57169577e-manager-config\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.607725 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.607705 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/5bed4249-443e-4f02-8ba2-22e57169577e-metrics-cert\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.607808 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.607769 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5bed4249-443e-4f02-8ba2-22e57169577e-cert\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.617520 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.617495 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhqnq\" (UniqueName: \"kubernetes.io/projected/5bed4249-443e-4f02-8ba2-22e57169577e-kube-api-access-lhqnq\") pod \"lws-controller-manager-7b4664dc78-msgg9\" (UID: \"5bed4249-443e-4f02-8ba2-22e57169577e\") " pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.801967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.801876 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:36.877838 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.877802 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:36.946652 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:36.946617 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9"] Jul 15 20:01:36.949639 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:01:36.949614 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bed4249_443e_4f02_8ba2_22e57169577e.slice/crio-a986eeaf648719e12101f876b5ba6f97a827422e9e0912e0fae0affa6931b2c9 WatchSource:0}: Error finding container a986eeaf648719e12101f876b5ba6f97a827422e9e0912e0fae0affa6931b2c9: Status 404 returned error can't find the container with id a986eeaf648719e12101f876b5ba6f97a827422e9e0912e0fae0affa6931b2c9 Jul 15 20:01:37.210215 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.210178 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx"] Jul 15 20:01:37.215249 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.215213 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.218548 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.218529 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 20:01:37.218639 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.218560 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-l8mgg\"" Jul 15 20:01:37.218859 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.218839 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 20:01:37.219600 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.219587 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 20:01:37.234412 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.234388 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx"] Jul 15 20:01:37.310510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310476 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.310510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310522 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.310786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310539 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.310786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310564 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.310786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310676 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.310786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310724 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.310786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310755 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbtlc\" (UniqueName: \"kubernetes.io/projected/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-kube-api-access-kbtlc\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.311053 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310794 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.311053 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.310874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.411776 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.411736 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412077 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.411802 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412077 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.411830 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412077 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.411853 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412287 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412178 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412287 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412226 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412418 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412291 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412418 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412330 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412418 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412346 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412418 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbtlc\" (UniqueName: \"kubernetes.io/projected/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-kube-api-access-kbtlc\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412418 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412404 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412665 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412435 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412750 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-data\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.412962 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.412944 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.414622 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.414598 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.414851 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.414835 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.421089 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.421064 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-istio-token\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.421222 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.421205 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbtlc\" (UniqueName: \"kubernetes.io/projected/c1928ce8-9c2f-4207-8581-e0b1fa3c1381-kube-api-access-kbtlc\") pod \"data-science-gateway-data-science-gateway-class-7f964fb678wh6nx\" (UID: \"c1928ce8-9c2f-4207-8581-e0b1fa3c1381\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.526122 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.526031 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:37.690111 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.690084 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx"] Jul 15 20:01:37.692366 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:01:37.692339 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1928ce8_9c2f_4207_8581_e0b1fa3c1381.slice/crio-aaa816a4a48ae329c50c49d78b252afc6c2ab2078151c4de7c94561ede16b4c4 WatchSource:0}: Error finding container aaa816a4a48ae329c50c49d78b252afc6c2ab2078151c4de7c94561ede16b4c4: Status 404 returned error can't find the container with id aaa816a4a48ae329c50c49d78b252afc6c2ab2078151c4de7c94561ede16b4c4 Jul 15 20:01:37.929328 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.929295 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:37.933006 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.932982 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" event={"ID":"c1928ce8-9c2f-4207-8581-e0b1fa3c1381","Type":"ContainerStarted","Data":"aaa816a4a48ae329c50c49d78b252afc6c2ab2078151c4de7c94561ede16b4c4"} Jul 15 20:01:37.934254 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:37.934216 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" event={"ID":"5bed4249-443e-4f02-8ba2-22e57169577e","Type":"ContainerStarted","Data":"a986eeaf648719e12101f876b5ba6f97a827422e9e0912e0fae0affa6931b2c9"} Jul 15 20:01:38.037919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:38.037882 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:38.951671 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:38.951562 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:39.105272 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.105207 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-psv5z"] Jul 15 20:01:39.109832 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.109746 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:39.110307 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.110283 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.124803 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.124775 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 20:01:39.125701 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.125451 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-2z787\"" Jul 15 20:01:39.130630 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.130601 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7c9gv\" (UniqueName: \"kubernetes.io/projected/5ddeb833-fb22-40bc-a670-8ff1c80c8197-kube-api-access-7c9gv\") pod \"kserve-controller-manager-d6f4b65b8-psv5z\" (UID: \"5ddeb833-fb22-40bc-a670-8ff1c80c8197\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.130755 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.130683 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ddeb833-fb22-40bc-a670-8ff1c80c8197-cert\") pod \"kserve-controller-manager-d6f4b65b8-psv5z\" (UID: \"5ddeb833-fb22-40bc-a670-8ff1c80c8197\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.148325 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.148293 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q"] Jul 15 20:01:39.154378 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.154350 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-psv5z"] Jul 15 20:01:39.154526 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.154474 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.158462 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.158005 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5b5tn\"" Jul 15 20:01:39.158462 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.158227 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 20:01:39.176961 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.175528 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q"] Jul 15 20:01:39.234919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.232118 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.234919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.232223 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7c9gv\" (UniqueName: \"kubernetes.io/projected/5ddeb833-fb22-40bc-a670-8ff1c80c8197-kube-api-access-7c9gv\") pod \"kserve-controller-manager-d6f4b65b8-psv5z\" (UID: \"5ddeb833-fb22-40bc-a670-8ff1c80c8197\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.234919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.232286 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55lnw\" (UniqueName: \"kubernetes.io/projected/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-kube-api-access-55lnw\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.234919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.232328 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ddeb833-fb22-40bc-a670-8ff1c80c8197-cert\") pod \"kserve-controller-manager-d6f4b65b8-psv5z\" (UID: \"5ddeb833-fb22-40bc-a670-8ff1c80c8197\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.242476 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.242444 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5ddeb833-fb22-40bc-a670-8ff1c80c8197-cert\") pod \"kserve-controller-manager-d6f4b65b8-psv5z\" (UID: \"5ddeb833-fb22-40bc-a670-8ff1c80c8197\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.259762 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.258912 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7c9gv\" (UniqueName: \"kubernetes.io/projected/5ddeb833-fb22-40bc-a670-8ff1c80c8197-kube-api-access-7c9gv\") pod \"kserve-controller-manager-d6f4b65b8-psv5z\" (UID: \"5ddeb833-fb22-40bc-a670-8ff1c80c8197\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.335285 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.333599 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55lnw\" (UniqueName: \"kubernetes.io/projected/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-kube-api-access-55lnw\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.335285 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.333690 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.335285 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:01:39.333829 2557 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found Jul 15 20:01:39.335285 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:01:39.333888 2557 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-cert podName:f0632d30-c3aa-4105-8f6b-4b7a7e4a5796 nodeName:}" failed. No retries permitted until 2026-07-15 20:01:39.833867254 +0000 UTC m=+447.504653774 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-cert") pod "llmisvc-controller-manager-777c6bd7d5-bb97q" (UID: "f0632d30-c3aa-4105-8f6b-4b7a7e4a5796") : secret "llmisvc-webhook-server-cert" not found Jul 15 20:01:39.335285 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.334763 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:39.348932 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.348900 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55lnw\" (UniqueName: \"kubernetes.io/projected/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-kube-api-access-55lnw\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.426328 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.426297 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:39.477160 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.477024 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:39.662536 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.662464 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-psv5z"] Jul 15 20:01:39.664506 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.664451 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:39.666554 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:01:39.666501 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ddeb833_fb22_40bc_a670_8ff1c80c8197.slice/crio-ae4a648762f1017987657044ccf5b4b2b36eac190d3423574332b2cf89d82a51 WatchSource:0}: Error finding container ae4a648762f1017987657044ccf5b4b2b36eac190d3423574332b2cf89d82a51: Status 404 returned error can't find the container with id ae4a648762f1017987657044ccf5b4b2b36eac190d3423574332b2cf89d82a51 Jul 15 20:01:39.840270 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.840173 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.842847 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.842826 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f0632d30-c3aa-4105-8f6b-4b7a7e4a5796-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-bb97q\" (UID: \"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:39.927625 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.927592 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:39.949229 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:39.949194 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" event={"ID":"5ddeb833-fb22-40bc-a670-8ff1c80c8197","Type":"ContainerStarted","Data":"ae4a648762f1017987657044ccf5b4b2b36eac190d3423574332b2cf89d82a51"} Jul 15 20:01:40.072763 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:40.072551 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:42.114048 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.113139 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:01:42.114048 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.113497 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:01:42.115661 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.114803 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:01:42.260638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.260603 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q"] Jul 15 20:01:42.263602 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:01:42.263553 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf0632d30_c3aa_4105_8f6b_4b7a7e4a5796.slice/crio-d3601843fc8432f8461b2ca9bdb14c80acf3e8faaccf4a1b0feaf6521630b28e WatchSource:0}: Error finding container d3601843fc8432f8461b2ca9bdb14c80acf3e8faaccf4a1b0feaf6521630b28e: Status 404 returned error can't find the container with id d3601843fc8432f8461b2ca9bdb14c80acf3e8faaccf4a1b0feaf6521630b28e Jul 15 20:01:42.290351 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.290308 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:42.395827 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.395708 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:42.968612 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.968567 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" event={"ID":"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796","Type":"ContainerStarted","Data":"d3601843fc8432f8461b2ca9bdb14c80acf3e8faaccf4a1b0feaf6521630b28e"} Jul 15 20:01:42.970726 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.970684 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" event={"ID":"c1928ce8-9c2f-4207-8581-e0b1fa3c1381","Type":"ContainerStarted","Data":"8192344d743e8087d6e584d228bc023363ed00f9f52194723a8943a479209a87"} Jul 15 20:01:42.972607 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.972577 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" event={"ID":"5bed4249-443e-4f02-8ba2-22e57169577e","Type":"ContainerStarted","Data":"18af3b1dc2a76f537bcab7fa1f1249c230fcc6e1759561c57789acf14ccb1d26"} Jul 15 20:01:42.972753 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.972731 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:42.997074 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:42.997021 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" podStartSLOduration=1.578852224 podStartE2EDuration="5.997004753s" totalImagesPullingTime="4.418152529s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:42.996450823 +0000 UTC m=+450.667237354" watchObservedRunningTime="2026-07-15 20:01:42.997004753 +0000 UTC m=+450.667791259" Jul 15 20:01:43.033305 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.033211 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" podStartSLOduration=1.884903129 podStartE2EDuration="7.033190684s" totalImagesPullingTime="5.148287555s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:43.031634841 +0000 UTC m=+450.702421391" watchObservedRunningTime="2026-07-15 20:01:43.033190684 +0000 UTC m=+450.703977215" Jul 15 20:01:43.526618 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.526585 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:43.531479 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.531455 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:43.980410 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.980367 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" event={"ID":"5ddeb833-fb22-40bc-a670-8ff1c80c8197","Type":"ContainerStarted","Data":"9038dcce95ca278ac42dccad5e6458c051fa865a7e10dfc86ca495c26b1662c5"} Jul 15 20:01:43.980631 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.980614 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:01:43.980893 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.980869 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:43.982093 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:43.982075 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-7f964fb678wh6nx" Jul 15 20:01:44.013459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:44.013407 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" podStartSLOduration=1.3828402579999999 podStartE2EDuration="5.013360077s" totalImagesPullingTime="3.630519819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:44.011123239 +0000 UTC m=+451.681909802" watchObservedRunningTime="2026-07-15 20:01:44.013360077 +0000 UTC m=+451.684146602" Jul 15 20:01:46.998410 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:46.998379 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" event={"ID":"f0632d30-c3aa-4105-8f6b-4b7a7e4a5796","Type":"ContainerStarted","Data":"805137addd5e676f935456f8990b1a54d7d0ad69c9319b4a4bb2e946063fc174"} Jul 15 20:01:46.998812 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:46.998606 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:01:47.049421 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:47.049369 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" podStartSLOduration=3.924961089 podStartE2EDuration="8.049350948s" totalImagesPullingTime="4.124389859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:01:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:01:47.045722527 +0000 UTC m=+454.716509055" watchObservedRunningTime="2026-07-15 20:01:47.049350948 +0000 UTC m=+454.720137472" Jul 15 20:01:53.983075 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:53.983045 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b4664dc78-msgg9" Jul 15 20:01:58.183126 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:58.183086 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:01:58.284149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:01:58.284110 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:14.991636 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:14.991605 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-psv5z" Jul 15 20:02:18.005158 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:18.005124 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-bb97q" Jul 15 20:02:25.649898 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:25.649852 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:25.775260 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:25.775210 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:27.621915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:27.621880 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:28.912621 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:28.912564 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:29.013129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.013095 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:29.118643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.118606 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-klmlg"] Jul 15 20:02:29.128598 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.128569 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:29.131529 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.131489 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-pv9mw\"" Jul 15 20:02:29.132702 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.132681 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 20:02:29.133009 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.132991 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 20:02:29.135051 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.135028 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-klmlg"] Jul 15 20:02:29.163145 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.163082 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:29.210852 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.210823 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hl7rv\" (UniqueName: \"kubernetes.io/projected/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7-kube-api-access-hl7rv\") pod \"kuadrant-operator-catalog-klmlg\" (UID: \"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7\") " pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:29.312351 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.312320 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hl7rv\" (UniqueName: \"kubernetes.io/projected/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7-kube-api-access-hl7rv\") pod \"kuadrant-operator-catalog-klmlg\" (UID: \"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7\") " pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:29.320584 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.320556 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hl7rv\" (UniqueName: \"kubernetes.io/projected/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7-kube-api-access-hl7rv\") pod \"kuadrant-operator-catalog-klmlg\" (UID: \"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7\") " pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:29.441790 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.441760 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:29.525906 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.525874 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-klmlg"] Jul 15 20:02:29.587374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.587228 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-klmlg"] Jul 15 20:02:29.590272 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:02:29.590230 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2444ac47_fccd_4d56_a64f_b3cf8d8f34d7.slice/crio-dc8bed5d9f6c051b8426faa6dab800690cffbf84c7e1984fe043334c3832a363 WatchSource:0}: Error finding container dc8bed5d9f6c051b8426faa6dab800690cffbf84c7e1984fe043334c3832a363: Status 404 returned error can't find the container with id dc8bed5d9f6c051b8426faa6dab800690cffbf84c7e1984fe043334c3832a363 Jul 15 20:02:29.675692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.675661 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7zbn"] Jul 15 20:02:29.678682 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.678665 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:29.689430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.689406 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7zbn"] Jul 15 20:02:29.715752 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.715686 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qf494\" (UniqueName: \"kubernetes.io/projected/c84a0ec2-070f-45fc-a1ac-f0c5456ad889-kube-api-access-qf494\") pod \"kuadrant-operator-catalog-s7zbn\" (UID: \"c84a0ec2-070f-45fc-a1ac-f0c5456ad889\") " pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:29.816998 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.816962 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qf494\" (UniqueName: \"kubernetes.io/projected/c84a0ec2-070f-45fc-a1ac-f0c5456ad889-kube-api-access-qf494\") pod \"kuadrant-operator-catalog-s7zbn\" (UID: \"c84a0ec2-070f-45fc-a1ac-f0c5456ad889\") " pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:29.825695 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.825660 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qf494\" (UniqueName: \"kubernetes.io/projected/c84a0ec2-070f-45fc-a1ac-f0c5456ad889-kube-api-access-qf494\") pod \"kuadrant-operator-catalog-s7zbn\" (UID: \"c84a0ec2-070f-45fc-a1ac-f0c5456ad889\") " pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:29.988199 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:29.988109 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:30.126924 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:30.126900 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-s7zbn"] Jul 15 20:02:30.128619 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:02:30.128591 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc84a0ec2_070f_45fc_a1ac_f0c5456ad889.slice/crio-9a0f3aca01f2a454fad32ab7f4a099ff101b7bdc21d8d582c947fce623ad78a8 WatchSource:0}: Error finding container 9a0f3aca01f2a454fad32ab7f4a099ff101b7bdc21d8d582c947fce623ad78a8: Status 404 returned error can't find the container with id 9a0f3aca01f2a454fad32ab7f4a099ff101b7bdc21d8d582c947fce623ad78a8 Jul 15 20:02:30.204300 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:30.204268 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" event={"ID":"c84a0ec2-070f-45fc-a1ac-f0c5456ad889","Type":"ContainerStarted","Data":"9a0f3aca01f2a454fad32ab7f4a099ff101b7bdc21d8d582c947fce623ad78a8"} Jul 15 20:02:30.205510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:30.205484 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" event={"ID":"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7","Type":"ContainerStarted","Data":"dc8bed5d9f6c051b8426faa6dab800690cffbf84c7e1984fe043334c3832a363"} Jul 15 20:02:32.217899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.217860 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" event={"ID":"c84a0ec2-070f-45fc-a1ac-f0c5456ad889","Type":"ContainerStarted","Data":"20bf1b7f60b677d75676eec2db0fb34a5c941807eab8f2d29061144f4659e869"} Jul 15 20:02:32.219249 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.219209 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" event={"ID":"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7","Type":"ContainerStarted","Data":"421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f"} Jul 15 20:02:32.219384 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.219301 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" podUID="2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" containerName="registry-server" containerID="cri-o://421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f" gracePeriod=2 Jul 15 20:02:32.240585 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.240542 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" podStartSLOduration=1.670727249 podStartE2EDuration="3.240524919s" totalImagesPullingTime="1.56979767s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:02:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:02:32.240003996 +0000 UTC m=+499.910790532" watchObservedRunningTime="2026-07-15 20:02:32.240524919 +0000 UTC m=+499.911311441" Jul 15 20:02:32.260641 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.260595 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" podStartSLOduration=1.152286839 podStartE2EDuration="3.260577648s" totalImagesPullingTime="2.108290809s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:02:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:02:32.25973901 +0000 UTC m=+499.930525534" watchObservedRunningTime="2026-07-15 20:02:32.260577648 +0000 UTC m=+499.931364175" Jul 15 20:02:32.475651 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.475581 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:32.542518 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.542485 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hl7rv\" (UniqueName: \"kubernetes.io/projected/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7-kube-api-access-hl7rv\") pod \"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7\" (UID: \"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7\") " Jul 15 20:02:32.544771 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.544748 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7-kube-api-access-hl7rv" (OuterVolumeSpecName: "kube-api-access-hl7rv") pod "2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" (UID: "2444ac47-fccd-4d56-a64f-b3cf8d8f34d7"). InnerVolumeSpecName "kube-api-access-hl7rv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:02:32.643821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:32.643779 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hl7rv\" (UniqueName: \"kubernetes.io/projected/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7-kube-api-access-hl7rv\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:02:33.225582 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.225547 2557 generic.go:356] "Generic (PLEG): container finished" podID="2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" containerID="421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f" exitCode=0 Jul 15 20:02:33.225998 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.225601 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" Jul 15 20:02:33.225998 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.225626 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" event={"ID":"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7","Type":"ContainerDied","Data":"421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f"} Jul 15 20:02:33.225998 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.225686 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-klmlg" event={"ID":"2444ac47-fccd-4d56-a64f-b3cf8d8f34d7","Type":"ContainerDied","Data":"dc8bed5d9f6c051b8426faa6dab800690cffbf84c7e1984fe043334c3832a363"} Jul 15 20:02:33.225998 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.225709 2557 scope.go:117] "RemoveContainer" containerID="421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f" Jul 15 20:02:33.234595 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.234577 2557 scope.go:117] "RemoveContainer" containerID="421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f" Jul 15 20:02:33.234851 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:02:33.234829 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f\": container with ID starting with 421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f not found: ID does not exist" containerID="421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f" Jul 15 20:02:33.234916 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.234860 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f"} err="failed to get container status \"421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f\": rpc error: code = NotFound desc = could not find container \"421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f\": container with ID starting with 421b8e845571a95705c70d9bcd997f6526f85c4759b5f60ab3b79b7dc2868a4f not found: ID does not exist" Jul 15 20:02:33.244117 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.244096 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-klmlg"] Jul 15 20:02:33.250959 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:33.250939 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-klmlg"] Jul 15 20:02:34.786074 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:34.786040 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" path="/var/lib/kubelet/pods/2444ac47-fccd-4d56-a64f-b3cf8d8f34d7/volumes" Jul 15 20:02:39.988449 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:39.988413 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:39.988922 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:39.988496 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:40.012265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:40.012223 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:40.282855 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:40.282769 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-s7zbn" Jul 15 20:02:53.739156 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:53.739110 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:56.858995 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:56.858958 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:58.734346 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:58.734313 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:59.131955 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:59.131870 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:02:59.919599 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:02:59.919568 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:00.031638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:00.031606 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:01.318223 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.318189 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt"] Jul 15 20:03:01.318676 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.318660 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" containerName="registry-server" Jul 15 20:03:01.318727 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.318678 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" containerName="registry-server" Jul 15 20:03:01.318762 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.318749 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2444ac47-fccd-4d56-a64f-b3cf8d8f34d7" containerName="registry-server" Jul 15 20:03:01.326373 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.326352 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:01.329057 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.329036 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-w7wck\"" Jul 15 20:03:01.336689 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.336668 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt"] Jul 15 20:03:01.412565 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.412532 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c56w\" (UniqueName: \"kubernetes.io/projected/b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f-kube-api-access-9c56w\") pod \"authorino-operator-75cc8d5cb7-mbxmt\" (UID: \"b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:01.513394 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.513363 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c56w\" (UniqueName: \"kubernetes.io/projected/b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f-kube-api-access-9c56w\") pod \"authorino-operator-75cc8d5cb7-mbxmt\" (UID: \"b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:01.523119 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.523083 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c56w\" (UniqueName: \"kubernetes.io/projected/b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f-kube-api-access-9c56w\") pod \"authorino-operator-75cc8d5cb7-mbxmt\" (UID: \"b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:01.638123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.638040 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:01.744144 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.744109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:01.783817 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:01.783794 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt"] Jul 15 20:03:01.785425 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:03:01.785404 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb8ccfe76_91f8_4ab3_96e6_5c1cdac2359f.slice/crio-a9d7e6a8612d72575bfa581e1d65d7047505d89e1368712cd904d77fe92efcba WatchSource:0}: Error finding container a9d7e6a8612d72575bfa581e1d65d7047505d89e1368712cd904d77fe92efcba: Status 404 returned error can't find the container with id a9d7e6a8612d72575bfa581e1d65d7047505d89e1368712cd904d77fe92efcba Jul 15 20:03:02.367617 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:02.367568 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" event={"ID":"b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f","Type":"ContainerStarted","Data":"a9d7e6a8612d72575bfa581e1d65d7047505d89e1368712cd904d77fe92efcba"} Jul 15 20:03:02.492310 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:02.492223 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:03.678866 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.678771 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm"] Jul 15 20:03:03.687181 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.687153 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:03.690635 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.690615 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-72trf\"" Jul 15 20:03:03.704780 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.704746 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm"] Jul 15 20:03:03.737442 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.737383 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-njjm8\" (UniqueName: \"kubernetes.io/projected/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b-kube-api-access-njjm8\") pod \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" (UID: \"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:03.838739 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.838700 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-njjm8\" (UniqueName: \"kubernetes.io/projected/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b-kube-api-access-njjm8\") pod \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" (UID: \"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:03.850027 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.849992 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-njjm8\" (UniqueName: \"kubernetes.io/projected/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b-kube-api-access-njjm8\") pod \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" (UID: \"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:03.961505 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.961421 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:03.998708 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:03.998677 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:04.155224 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:04.155198 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm"] Jul 15 20:03:04.157396 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:03:04.157361 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1629edac_4cfd_4a34_a3ff_09a69ac0ab9b.slice/crio-0a726311c1e4e54a232602dba53b713bceb83459cfca1464f577eee14158f3a7 WatchSource:0}: Error finding container 0a726311c1e4e54a232602dba53b713bceb83459cfca1464f577eee14158f3a7: Status 404 returned error can't find the container with id 0a726311c1e4e54a232602dba53b713bceb83459cfca1464f577eee14158f3a7 Jul 15 20:03:04.386961 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:04.386865 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" event={"ID":"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b","Type":"ContainerStarted","Data":"0a726311c1e4e54a232602dba53b713bceb83459cfca1464f577eee14158f3a7"} Jul 15 20:03:04.388694 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:04.388660 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" event={"ID":"b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f","Type":"ContainerStarted","Data":"6846c46492f8caf09476b76b5305a6242b2e7281fba3c0959a57091e0583dfc2"} Jul 15 20:03:04.388868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:04.388763 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:04.409748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:04.409697 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" podStartSLOduration=1.172250558 podStartE2EDuration="3.409680108s" totalImagesPullingTime="2.23742955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:04.409363808 +0000 UTC m=+532.080150358" watchObservedRunningTime="2026-07-15 20:03:04.409680108 +0000 UTC m=+532.080466632" Jul 15 20:03:04.619307 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:04.619270 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:06.400888 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:06.400793 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" event={"ID":"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b","Type":"ContainerStarted","Data":"f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6"} Jul 15 20:03:06.401274 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:06.400899 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:06.420254 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:06.420189 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" podStartSLOduration=1.5143428669999999 podStartE2EDuration="3.420174509s" totalImagesPullingTime="1.905831642s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:06.41892658 +0000 UTC m=+534.089713109" watchObservedRunningTime="2026-07-15 20:03:06.420174509 +0000 UTC m=+534.090961034" Jul 15 20:03:07.638774 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:07.638742 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:07.798229 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:07.798181 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:08.670400 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:08.670363 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:08.872559 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:08.872530 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:15.397494 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:15.397464 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-mbxmt" Jul 15 20:03:17.407790 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:17.407758 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:17.455573 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:17.455525 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:17.552261 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:17.551803 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:17.669054 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:17.668966 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:17.907758 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:17.907727 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:20.128168 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.128137 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm"] Jul 15 20:03:20.128612 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.128346 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" containerName="manager" containerID="cri-o://f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6" gracePeriod=2 Jul 15 20:03:20.136107 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.136083 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm"] Jul 15 20:03:20.153516 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.153491 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss"] Jul 15 20:03:20.154135 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.154120 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" containerName="manager" Jul 15 20:03:20.154180 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.154143 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" containerName="manager" Jul 15 20:03:20.154249 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.154225 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" containerName="manager" Jul 15 20:03:20.157686 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.157666 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:20.160165 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:03:20.160138 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:20.166902 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.166879 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss"] Jul 15 20:03:20.297958 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.297930 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmw8h\" (UniqueName: \"kubernetes.io/projected/1a27705c-072f-4fa1-8659-764e557b6759-kube-api-access-gmw8h\") pod \"limitador-operator-controller-manager-859d7b78bd-v5dss\" (UID: \"1a27705c-072f-4fa1-8659-764e557b6759\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:20.374545 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.374522 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:20.376933 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:03:20.376909 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:20.398851 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.398784 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmw8h\" (UniqueName: \"kubernetes.io/projected/1a27705c-072f-4fa1-8659-764e557b6759-kube-api-access-gmw8h\") pod \"limitador-operator-controller-manager-859d7b78bd-v5dss\" (UID: \"1a27705c-072f-4fa1-8659-764e557b6759\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:20.405278 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.404841 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:20.409415 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.409382 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmw8h\" (UniqueName: \"kubernetes.io/projected/1a27705c-072f-4fa1-8659-764e557b6759-kube-api-access-gmw8h\") pod \"limitador-operator-controller-manager-859d7b78bd-v5dss\" (UID: \"1a27705c-072f-4fa1-8659-764e557b6759\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:20.470401 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.470373 2557 generic.go:356] "Generic (PLEG): container finished" podID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" containerID="f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6" exitCode=0 Jul 15 20:03:20.470525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.470433 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:20.470525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.470476 2557 scope.go:117] "RemoveContainer" containerID="f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6" Jul 15 20:03:20.472807 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:03:20.472785 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:20.479171 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.479149 2557 scope.go:117] "RemoveContainer" containerID="f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6" Jul 15 20:03:20.479432 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:03:20.479413 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6\": container with ID starting with f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6 not found: ID does not exist" containerID="f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6" Jul 15 20:03:20.479501 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.479440 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6"} err="failed to get container status \"f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6\": rpc error: code = NotFound desc = could not find container \"f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6\": container with ID starting with f8557b7ed02cc3929e25d56c250496b4d9c9af298b96baebfab06f5db9f418b6 not found: ID does not exist" Jul 15 20:03:20.498458 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.498433 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:20.499526 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.499507 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-njjm8\" (UniqueName: \"kubernetes.io/projected/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b-kube-api-access-njjm8\") pod \"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b\" (UID: \"1629edac-4cfd-4a34-a3ff-09a69ac0ab9b\") " Jul 15 20:03:20.501635 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.501616 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b-kube-api-access-njjm8" (OuterVolumeSpecName: "kube-api-access-njjm8") pod "1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" (UID: "1629edac-4cfd-4a34-a3ff-09a69ac0ab9b"). InnerVolumeSpecName "kube-api-access-njjm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:03:20.502714 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.502699 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:20.600864 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.600834 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-njjm8\" (UniqueName: \"kubernetes.io/projected/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b-kube-api-access-njjm8\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:03:20.668423 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.668275 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss"] Jul 15 20:03:20.670923 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:03:20.670896 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a27705c_072f_4fa1_8659_764e557b6759.slice/crio-9fbe0791324ab34038b54e79c4fde77e0f8c0df98a7a019f9e36a7ef68133384 WatchSource:0}: Error finding container 9fbe0791324ab34038b54e79c4fde77e0f8c0df98a7a019f9e36a7ef68133384: Status 404 returned error can't find the container with id 9fbe0791324ab34038b54e79c4fde77e0f8c0df98a7a019f9e36a7ef68133384 Jul 15 20:03:20.781864 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:03:20.781834 2557 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-4dgxm\" is forbidden: User \"system:node:ip-10-0-132-12.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-12.ec2.internal' and this object" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-4dgxm" Jul 15 20:03:20.785805 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:20.785782 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1629edac-4cfd-4a34-a3ff-09a69ac0ab9b" path="/var/lib/kubelet/pods/1629edac-4cfd-4a34-a3ff-09a69ac0ab9b/volumes" Jul 15 20:03:21.477656 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:21.477622 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" event={"ID":"1a27705c-072f-4fa1-8659-764e557b6759","Type":"ContainerStarted","Data":"c1db6c8e48629df21eaaf128ccaa42e181923bb9382879cf6a17574b71a854d9"} Jul 15 20:03:21.478035 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:21.477675 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" event={"ID":"1a27705c-072f-4fa1-8659-764e557b6759","Type":"ContainerStarted","Data":"9fbe0791324ab34038b54e79c4fde77e0f8c0df98a7a019f9e36a7ef68133384"} Jul 15 20:03:21.478035 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:21.477708 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:21.502662 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:21.502603 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" podStartSLOduration=1.5025853919999999 podStartE2EDuration="1.502585392s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:21.501611647 +0000 UTC m=+549.172398171" watchObservedRunningTime="2026-07-15 20:03:21.502585392 +0000 UTC m=+549.173371980" Jul 15 20:03:22.848803 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:22.848772 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:22.948456 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:22.948418 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:28.848823 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:28.848790 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:28.948080 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:28.948045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:29.050564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:29.050531 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:32.485215 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:32.485187 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-v5dss" Jul 15 20:03:37.810496 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:37.810459 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:37.909855 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:37.909817 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:38.011831 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:38.011802 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:40.875472 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:40.875432 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:41.024576 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:41.024544 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:43.244200 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:43.244166 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:43.343951 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:43.343903 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:49.705163 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.705127 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx"] Jul 15 20:03:49.710177 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.710154 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.713821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.713795 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mdkq5\"" Jul 15 20:03:49.726659 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.726630 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx"] Jul 15 20:03:49.771911 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.771880 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.771911 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.771915 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772114 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.771939 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5690746-549e-4381-85cb-a74a922f05e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772114 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.771962 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772114 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.772008 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gp9pj\" (UniqueName: \"kubernetes.io/projected/c5690746-549e-4381-85cb-a74a922f05e3-kube-api-access-gp9pj\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772114 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.772082 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5690746-549e-4381-85cb-a74a922f05e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772114 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.772105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772357 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.772130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.772357 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.772164 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5690746-549e-4381-85cb-a74a922f05e3-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873296 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5690746-549e-4381-85cb-a74a922f05e3-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873368 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873421 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873453 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5690746-549e-4381-85cb-a74a922f05e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873487 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873534 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gp9pj\" (UniqueName: \"kubernetes.io/projected/c5690746-549e-4381-85cb-a74a922f05e3-kube-api-access-gp9pj\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873840 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873630 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5690746-549e-4381-85cb-a74a922f05e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873840 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873656 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.873840 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873684 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.874000 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873918 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-workload-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.874000 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.873945 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-workload-certs\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.874564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.874534 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-istio-data\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.874690 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.874561 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-credential-socket\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.875019 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.875000 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c5690746-549e-4381-85cb-a74a922f05e3-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.876628 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.876605 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c5690746-549e-4381-85cb-a74a922f05e3-istio-envoy\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.876946 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.876924 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c5690746-549e-4381-85cb-a74a922f05e3-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.882704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.882680 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c5690746-549e-4381-85cb-a74a922f05e3-istio-token\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:49.882795 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:49.882714 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gp9pj\" (UniqueName: \"kubernetes.io/projected/c5690746-549e-4381-85cb-a74a922f05e3-kube-api-access-gp9pj\") pod \"maas-default-gateway-openshift-default-5896d4dd49-c5jhx\" (UID: \"c5690746-549e-4381-85cb-a74a922f05e3\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:50.021685 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.021607 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:50.051431 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.051404 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:50.166136 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.166105 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx"] Jul 15 20:03:50.167982 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:03:50.167943 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5690746_549e_4381_85cb_a74a922f05e3.slice/crio-1190ce0bddf210d363db183829a0de9258e693a4da313cda0e04411dfb672232 WatchSource:0}: Error finding container 1190ce0bddf210d363db183829a0de9258e693a4da313cda0e04411dfb672232: Status 404 returned error can't find the container with id 1190ce0bddf210d363db183829a0de9258e693a4da313cda0e04411dfb672232 Jul 15 20:03:50.170150 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.170117 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:03:50.170254 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.170197 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:03:50.170254 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.170225 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:03:50.621692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.621657 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" event={"ID":"c5690746-549e-4381-85cb-a74a922f05e3","Type":"ContainerStarted","Data":"e71741bb510fab449a9dd3b11135227528a656609e369af3019438eebd93d988"} Jul 15 20:03:50.621861 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.621712 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" event={"ID":"c5690746-549e-4381-85cb-a74a922f05e3","Type":"ContainerStarted","Data":"1190ce0bddf210d363db183829a0de9258e693a4da313cda0e04411dfb672232"} Jul 15 20:03:50.643835 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.643784 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" podStartSLOduration=1.6437646510000001 podStartE2EDuration="1.643764651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:03:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:03:50.642769695 +0000 UTC m=+578.313556239" watchObservedRunningTime="2026-07-15 20:03:50.643764651 +0000 UTC m=+578.314551175" Jul 15 20:03:50.876058 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:50.875974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:51.022653 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.022621 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:51.027834 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.027806 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:51.626513 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.626482 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:51.627656 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.627634 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:03:51.703553 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.703522 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:51.810583 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.810550 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:51.939158 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:51.939123 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:52.560695 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:52.560658 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:03:52.659672 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:03:52.659639 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:07.867593 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:07.867559 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:08.018127 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:08.018093 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:08.349084 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:08.349050 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:19.861787 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:19.861753 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:20.119593 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:20.119506 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:26.383778 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:26.383748 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:26.465601 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:26.465565 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:26.542884 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:26.542846 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:29.659811 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:29.659773 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:29.797565 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:29.797533 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:44.882429 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:44.882380 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:56.886486 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:56.886444 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:04:56.964338 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:04:56.964309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:01.034305 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:01.034268 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:01.691815 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:01.691783 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:01.948895 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:01.948805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:02.151216 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:02.151184 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:02.659280 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:02.659224 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:02.932913 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:02.932883 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:03.007638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.007609 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9"] Jul 15 20:05:03.011295 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.011271 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.013779 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.013752 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-68n9b\"" Jul 15 20:05:03.013962 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.013944 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 20:05:03.049426 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.049391 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9"] Jul 15 20:05:03.067835 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.067809 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-xln9v"] Jul 15 20:05:03.071378 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.071360 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.076896 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.076870 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dcbf538-3f10-4aca-a98c-429c13a73ca2-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-rqlj9\" (UID: \"8dcbf538-3f10-4aca-a98c-429c13a73ca2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.077041 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.077020 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lfjj\" (UniqueName: \"kubernetes.io/projected/8dcbf538-3f10-4aca-a98c-429c13a73ca2-kube-api-access-9lfjj\") pod \"payload-pre-processing-74584bb9f8-rqlj9\" (UID: \"8dcbf538-3f10-4aca-a98c-429c13a73ca2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.077099 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.077054 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b159e8-12be-4899-8aa8-55fb08f88476-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-xln9v\" (UID: \"73b159e8-12be-4899-8aa8-55fb08f88476\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.077099 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.077087 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rhx6\" (UniqueName: \"kubernetes.io/projected/73b159e8-12be-4899-8aa8-55fb08f88476-kube-api-access-8rhx6\") pod \"payload-processing-f9b8d9b9b-xln9v\" (UID: \"73b159e8-12be-4899-8aa8-55fb08f88476\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.094133 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.094106 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:03.094351 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.094324 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-xln9v"] Jul 15 20:05:03.178547 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.178508 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dcbf538-3f10-4aca-a98c-429c13a73ca2-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-rqlj9\" (UID: \"8dcbf538-3f10-4aca-a98c-429c13a73ca2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.178967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.178618 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lfjj\" (UniqueName: \"kubernetes.io/projected/8dcbf538-3f10-4aca-a98c-429c13a73ca2-kube-api-access-9lfjj\") pod \"payload-pre-processing-74584bb9f8-rqlj9\" (UID: \"8dcbf538-3f10-4aca-a98c-429c13a73ca2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.178967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.178653 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b159e8-12be-4899-8aa8-55fb08f88476-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-xln9v\" (UID: \"73b159e8-12be-4899-8aa8-55fb08f88476\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.178967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.178689 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rhx6\" (UniqueName: \"kubernetes.io/projected/73b159e8-12be-4899-8aa8-55fb08f88476-kube-api-access-8rhx6\") pod \"payload-processing-f9b8d9b9b-xln9v\" (UID: \"73b159e8-12be-4899-8aa8-55fb08f88476\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.179327 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.179309 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8dcbf538-3f10-4aca-a98c-429c13a73ca2-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-rqlj9\" (UID: \"8dcbf538-3f10-4aca-a98c-429c13a73ca2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.179369 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.179348 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/73b159e8-12be-4899-8aa8-55fb08f88476-plugins-config-volume\") pod \"payload-processing-f9b8d9b9b-xln9v\" (UID: \"73b159e8-12be-4899-8aa8-55fb08f88476\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.189510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.189444 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lfjj\" (UniqueName: \"kubernetes.io/projected/8dcbf538-3f10-4aca-a98c-429c13a73ca2-kube-api-access-9lfjj\") pod \"payload-pre-processing-74584bb9f8-rqlj9\" (UID: \"8dcbf538-3f10-4aca-a98c-429c13a73ca2\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.189510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.189473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rhx6\" (UniqueName: \"kubernetes.io/projected/73b159e8-12be-4899-8aa8-55fb08f88476-kube-api-access-8rhx6\") pod \"payload-processing-f9b8d9b9b-xln9v\" (UID: \"73b159e8-12be-4899-8aa8-55fb08f88476\") " pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.322087 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.322048 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:03.381366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.381289 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:03.477801 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.477777 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9"] Jul 15 20:05:03.479382 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:05:03.479349 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dcbf538_3f10_4aca_a98c_429c13a73ca2.slice/crio-db84c1f1102f7960d7dd007ce8c42aeaf2a14be1189abaacba06e0c0b27bd69a WatchSource:0}: Error finding container db84c1f1102f7960d7dd007ce8c42aeaf2a14be1189abaacba06e0c0b27bd69a: Status 404 returned error can't find the container with id db84c1f1102f7960d7dd007ce8c42aeaf2a14be1189abaacba06e0c0b27bd69a Jul 15 20:05:03.539060 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.538985 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-f9b8d9b9b-xln9v"] Jul 15 20:05:03.541782 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:05:03.541750 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73b159e8_12be_4899_8aa8_55fb08f88476.slice/crio-7f41f609af041d568addebaea1cad16b495d87a0f6496cbe76a2bdf257a9d562 WatchSource:0}: Error finding container 7f41f609af041d568addebaea1cad16b495d87a0f6496cbe76a2bdf257a9d562: Status 404 returned error can't find the container with id 7f41f609af041d568addebaea1cad16b495d87a0f6496cbe76a2bdf257a9d562 Jul 15 20:05:03.988251 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.988198 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" event={"ID":"73b159e8-12be-4899-8aa8-55fb08f88476","Type":"ContainerStarted","Data":"7f41f609af041d568addebaea1cad16b495d87a0f6496cbe76a2bdf257a9d562"} Jul 15 20:05:03.989337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:03.989312 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" event={"ID":"8dcbf538-3f10-4aca-a98c-429c13a73ca2","Type":"ContainerStarted","Data":"db84c1f1102f7960d7dd007ce8c42aeaf2a14be1189abaacba06e0c0b27bd69a"} Jul 15 20:05:05.353042 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:05.353015 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:06.004519 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.004484 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" event={"ID":"8dcbf538-3f10-4aca-a98c-429c13a73ca2","Type":"ContainerStarted","Data":"5247a4f9e9d70c75bc66101711af959598fb34ebd84240b5a15313e1227efa04"} Jul 15 20:05:06.004769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.004558 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:06.006064 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.006026 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" event={"ID":"73b159e8-12be-4899-8aa8-55fb08f88476","Type":"ContainerStarted","Data":"3f54b2f6fb6682ede5f5a85fc6c3c8bf838848e6dd84d48f977b87b4f945a28f"} Jul 15 20:05:06.006189 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.006152 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:06.057262 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.054714 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" podStartSLOduration=2.279393206 podStartE2EDuration="4.054693653s" totalImagesPullingTime="1.775300447s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:06.032499181 +0000 UTC m=+653.703285707" watchObservedRunningTime="2026-07-15 20:05:06.054693653 +0000 UTC m=+653.725480179" Jul 15 20:05:06.057262 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.055097 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" podStartSLOduration=1.344319809 podStartE2EDuration="3.055089444s" totalImagesPullingTime="1.710769635s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:06.052844269 +0000 UTC m=+653.723630795" watchObservedRunningTime="2026-07-15 20:05:06.055089444 +0000 UTC m=+653.725875969" Jul 15 20:05:06.062073 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:06.062037 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:11.036777 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:11.036686 2557 generic.go:356] "Generic (PLEG): container finished" podID="c5690746-549e-4381-85cb-a74a922f05e3" containerID="e71741bb510fab449a9dd3b11135227528a656609e369af3019438eebd93d988" exitCode=0 Jul 15 20:05:11.036777 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:11.036767 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" event={"ID":"c5690746-549e-4381-85cb-a74a922f05e3","Type":"ContainerDied","Data":"e71741bb510fab449a9dd3b11135227528a656609e369af3019438eebd93d988"} Jul 15 20:05:11.037187 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:11.037172 2557 scope.go:117] "RemoveContainer" containerID="e71741bb510fab449a9dd3b11135227528a656609e369af3019438eebd93d988" Jul 15 20:05:11.038256 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:11.038201 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:05:11.038323 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:11.038298 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:05:11.038360 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:11.038331 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:05:12.044823 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:12.044786 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" event={"ID":"c5690746-549e-4381-85cb-a74a922f05e3","Type":"ContainerStarted","Data":"fcfdcdee509c93176cea3980f7d363c724a5c65a49c3fd5f0ac4deb67f9a32bd"} Jul 15 20:05:12.768834 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:12.768805 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:12.868406 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:12.868367 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:13.026555 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:13.026447 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:05:13.026875 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:13.026846 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-c5jhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 20:05:13.026964 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:13.026928 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" podUID="c5690746-549e-4381-85cb-a74a922f05e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 20:05:14.022532 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:14.022493 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-c5jhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 20:05:14.023132 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:14.023100 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" podUID="c5690746-549e-4381-85cb-a74a922f05e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 20:05:15.023139 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:15.023092 2557 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5896d4dd49-c5jhx container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" start-of-body= Jul 15 20:05:15.023749 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:15.023170 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" podUID="c5690746-549e-4381-85cb-a74a922f05e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.37:15021/healthz/ready\": dial tcp 10.132.0.37:15021: connect: connection refused" Jul 15 20:05:16.025550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:16.025518 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:05:16.026078 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:16.025800 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:05:16.026427 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:16.026409 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5896d4dd49-c5jhx" Jul 15 20:05:17.015451 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:17.015416 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-f9b8d9b9b-xln9v" Jul 15 20:05:17.015645 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:17.015623 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-rqlj9" Jul 15 20:05:30.659486 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:30.659441 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:41.690824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:41.690785 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:43.023613 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:43.023574 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:51.183614 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.183564 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv"] Jul 15 20:05:51.187445 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.187423 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.190037 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.190015 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 20:05:51.190982 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.190966 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-9fcf5\"" Jul 15 20:05:51.191079 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.191010 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 15 20:05:51.191079 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.191040 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 20:05:51.198290 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.198227 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv"] Jul 15 20:05:51.238088 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.238052 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:05:51.276572 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.276542 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.276765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.276586 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.276765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.276613 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qcmjv\" (UniqueName: \"kubernetes.io/projected/978ecb18-4960-428c-a75a-4df003f0014e-kube-api-access-qcmjv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.276765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.276643 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.276765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.276685 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978ecb18-4960-428c-a75a-4df003f0014e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.276765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.276709 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377497 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377457 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377699 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377512 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377699 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377544 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qcmjv\" (UniqueName: \"kubernetes.io/projected/978ecb18-4960-428c-a75a-4df003f0014e-kube-api-access-qcmjv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377699 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377570 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377699 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377621 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978ecb18-4960-428c-a75a-4df003f0014e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377699 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377658 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.377974 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.377950 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.378018 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.378001 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.378179 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.378156 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.379928 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.379909 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/978ecb18-4960-428c-a75a-4df003f0014e-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.380184 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.380167 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/978ecb18-4960-428c-a75a-4df003f0014e-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.394186 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.394152 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qcmjv\" (UniqueName: \"kubernetes.io/projected/978ecb18-4960-428c-a75a-4df003f0014e-kube-api-access-qcmjv\") pod \"e2e-trlp-test-simulated-kserve-598466548d-nb4cv\" (UID: \"978ecb18-4960-428c-a75a-4df003f0014e\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.498383 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.498281 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:05:51.648095 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.648073 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv"] Jul 15 20:05:51.649851 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:05:51.649824 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod978ecb18_4960_428c_a75a_4df003f0014e.slice/crio-bdbc72e58dde827bc54242caa806863a9ae731ef69e467ef50951bdfbcf6d10f WatchSource:0}: Error finding container bdbc72e58dde827bc54242caa806863a9ae731ef69e467ef50951bdfbcf6d10f: Status 404 returned error can't find the container with id bdbc72e58dde827bc54242caa806863a9ae731ef69e467ef50951bdfbcf6d10f Jul 15 20:05:51.651282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:51.651263 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:05:52.267708 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:52.267666 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" event={"ID":"978ecb18-4960-428c-a75a-4df003f0014e","Type":"ContainerStarted","Data":"bdbc72e58dde827bc54242caa806863a9ae731ef69e467ef50951bdfbcf6d10f"} Jul 15 20:05:53.274947 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:53.274909 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" event={"ID":"978ecb18-4960-428c-a75a-4df003f0014e","Type":"ContainerStarted","Data":"0776818dc51f55f3b61545ab5d9fcafea4a2d54d31cf39004f9306a0a5384435"} Jul 15 20:05:53.295856 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:05:53.295756 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" podStartSLOduration=0.902347885 podStartE2EDuration="2.29573884s" totalImagesPullingTime="1.393390955s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:05:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:05:53.293141471 +0000 UTC m=+700.963928011" watchObservedRunningTime="2026-07-15 20:05:53.29573884 +0000 UTC m=+700.966525364" Jul 15 20:06:01.498669 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:01.498617 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:06:01.499088 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:01.498691 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:06:01.513004 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:01.512974 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:06:02.332727 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:02.332698 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-nb4cv" Jul 15 20:06:04.795630 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:04.795539 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:06:12.296475 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:12.296436 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:06:17.685075 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:06:17.685042 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:07:11.055972 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:07:11.055928 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:07:12.040029 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:07:12.039991 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:07:12.149997 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:07:12.149964 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:07:13.211554 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:07:13.211526 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:07:13.326407 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:07:13.326366 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:19.717097 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:19.717060 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:19.870946 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:19.870909 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:20.400809 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.400774 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6c5bd6f84f-l9jqn"] Jul 15 20:08:20.404372 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.404354 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.408051 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408023 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 20:08:20.408051 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408040 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 20:08:20.408282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408023 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 20:08:20.408282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408078 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 20:08:20.408282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408022 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 20:08:20.408282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408049 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-4lxl7\"" Jul 15 20:08:20.408506 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408327 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 20:08:20.408506 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.408343 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 20:08:20.412667 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.412651 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 20:08:20.439100 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.439075 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c5bd6f84f-l9jqn"] Jul 15 20:08:20.453159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453130 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-serving-cert\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.453316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453185 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-oauth-serving-cert\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.453316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453263 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-service-ca\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.453316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453290 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f59dj\" (UniqueName: \"kubernetes.io/projected/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-kube-api-access-f59dj\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.453316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453312 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-oauth-config\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.453560 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453414 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-trusted-ca-bundle\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.453560 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.453446 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-config\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555010 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.554975 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-oauth-serving-cert\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555210 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555028 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-service-ca\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555210 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f59dj\" (UniqueName: \"kubernetes.io/projected/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-kube-api-access-f59dj\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555210 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555157 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-oauth-config\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555433 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555324 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-trusted-ca-bundle\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555433 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555374 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-config\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555539 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555447 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-serving-cert\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555793 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555773 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-oauth-serving-cert\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.555888 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.555867 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-service-ca\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.556157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.556136 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-config\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.556431 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.556416 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-trusted-ca-bundle\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.558516 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.558493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-oauth-config\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.558516 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.558504 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-console-serving-cert\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.566575 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.566553 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f59dj\" (UniqueName: \"kubernetes.io/projected/8e3381e0-5e8c-43dd-bd86-eb3f8e200468-kube-api-access-f59dj\") pod \"console-6c5bd6f84f-l9jqn\" (UID: \"8e3381e0-5e8c-43dd-bd86-eb3f8e200468\") " pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.715158 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.715129 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:20.861993 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:20.861965 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6c5bd6f84f-l9jqn"] Jul 15 20:08:20.864147 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:08:20.864110 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e3381e0_5e8c_43dd_bd86_eb3f8e200468.slice/crio-5d6c1af2f41bd42f93446a5e8cf8d82469ec7327a021cd8a0ab573d15bdacfdf WatchSource:0}: Error finding container 5d6c1af2f41bd42f93446a5e8cf8d82469ec7327a021cd8a0ab573d15bdacfdf: Status 404 returned error can't find the container with id 5d6c1af2f41bd42f93446a5e8cf8d82469ec7327a021cd8a0ab573d15bdacfdf Jul 15 20:08:21.060535 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:21.060446 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5bd6f84f-l9jqn" event={"ID":"8e3381e0-5e8c-43dd-bd86-eb3f8e200468","Type":"ContainerStarted","Data":"5d6c1af2f41bd42f93446a5e8cf8d82469ec7327a021cd8a0ab573d15bdacfdf"} Jul 15 20:08:24.079166 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:24.079126 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6c5bd6f84f-l9jqn" event={"ID":"8e3381e0-5e8c-43dd-bd86-eb3f8e200468","Type":"ContainerStarted","Data":"9e28bec791272af8587ed99e5c0906d465318e7d5bdbdda161182e3f97665829"} Jul 15 20:08:24.102324 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:24.102274 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6c5bd6f84f-l9jqn" podStartSLOduration=2.010713678 podStartE2EDuration="5.102255311s" totalImagesPullingTime="3.091541633s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:08:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:08:24.099937513 +0000 UTC m=+851.770724054" watchObservedRunningTime="2026-07-15 20:08:24.102255311 +0000 UTC m=+851.773041830" Jul 15 20:08:25.520770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:25.520730 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:27.501074 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:27.501038 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:30.715417 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:30.715377 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:30.715810 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:30.715448 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:30.716899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:30.716877 2557 patch_prober.go:28] interesting pod/console-6c5bd6f84f-l9jqn container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.41:8443/health\": dial tcp 10.132.0.41:8443: connect: connection refused" start-of-body= Jul 15 20:08:30.716979 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:30.716961 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-6c5bd6f84f-l9jqn" podUID="8e3381e0-5e8c-43dd-bd86-eb3f8e200468" containerName="console" probeResult="failure" output="Get \"https://10.132.0.41:8443/health\": dial tcp 10.132.0.41:8443: connect: connection refused" Jul 15 20:08:34.429192 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:34.429159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:40.719865 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:40.719831 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:40.723615 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:40.723593 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6c5bd6f84f-l9jqn" Jul 15 20:08:40.964340 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:40.964299 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:08:41.067151 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:08:41.067047 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:07.246921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:07.246832 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:08.479288 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:08.479257 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:08.584174 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:08.584140 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:14.686433 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:14.686400 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:14.798023 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:14.797986 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:14.886748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:14.886714 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:15.014347 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:15.014272 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:21.055861 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:21.055811 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:21.155608 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:21.155572 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:09:54.611011 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:09:54.610969 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:11:46.253129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:11:46.253096 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:11:47.867758 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:11:47.867721 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:14:35.681391 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:14:35.681351 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:14:35.746652 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:14:35.746613 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:14:42.856767 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:14:42.856726 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:15:00.145321 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.145286 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt"] Jul 15 20:15:00.149136 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.149116 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" Jul 15 20:15:00.151704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.151686 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:15:00.158851 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.158828 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt"] Jul 15 20:15:00.160785 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.160762 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" Jul 15 20:15:00.313995 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.313942 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt"] Jul 15 20:15:00.315812 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:15:00.315779 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07bb68c9_fde9_44f1_bc56_684e8031e6cf.slice/crio-a9df6e8cb52b2d52d94d363b9964ec06bbc9a51d2c5435433fffa77aa3804208 WatchSource:0}: Error finding container a9df6e8cb52b2d52d94d363b9964ec06bbc9a51d2c5435433fffa77aa3804208: Status 404 returned error can't find the container with id a9df6e8cb52b2d52d94d363b9964ec06bbc9a51d2c5435433fffa77aa3804208 Jul 15 20:15:00.317785 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:00.317766 2557 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 20:15:01.152854 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:01.152812 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" event={"ID":"07bb68c9-fde9-44f1-bc56-684e8031e6cf","Type":"ContainerStarted","Data":"a9df6e8cb52b2d52d94d363b9964ec06bbc9a51d2c5435433fffa77aa3804208"} Jul 15 20:15:03.170155 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:03.170116 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" event={"ID":"07bb68c9-fde9-44f1-bc56-684e8031e6cf","Type":"ContainerStarted","Data":"d8d4ffc322261905c369f7c2d3aec7eb80bc8b923d4606021047110c3d485dbe"} Jul 15 20:15:03.187473 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:03.187422 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" podStartSLOduration=1.071346478 podStartE2EDuration="3.187405282s" totalImagesPullingTime="2.116058804s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:15:03.18630099 +0000 UTC m=+1250.857087516" watchObservedRunningTime="2026-07-15 20:15:03.187405282 +0000 UTC m=+1250.858191806" Jul 15 20:15:04.176501 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:04.176463 2557 generic.go:356] "Generic (PLEG): container finished" podID="07bb68c9-fde9-44f1-bc56-684e8031e6cf" containerID="d8d4ffc322261905c369f7c2d3aec7eb80bc8b923d4606021047110c3d485dbe" exitCode=0 Jul 15 20:15:04.176958 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:04.176556 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" event={"ID":"07bb68c9-fde9-44f1-bc56-684e8031e6cf","Type":"ContainerDied","Data":"d8d4ffc322261905c369f7c2d3aec7eb80bc8b923d4606021047110c3d485dbe"} Jul 15 20:15:05.331093 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:05.331065 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" Jul 15 20:15:06.189579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:06.189544 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" event={"ID":"07bb68c9-fde9-44f1-bc56-684e8031e6cf","Type":"ContainerDied","Data":"a9df6e8cb52b2d52d94d363b9964ec06bbc9a51d2c5435433fffa77aa3804208"} Jul 15 20:15:06.189579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:06.189572 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735775-hz8gt" Jul 15 20:15:06.189579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:15:06.189594 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9df6e8cb52b2d52d94d363b9964ec06bbc9a51d2c5435433fffa77aa3804208" Jul 15 20:17:46.233703 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:17:46.233662 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:17:55.586275 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:17:55.582953 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:19:57.807561 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:19:57.807528 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:07.413489 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:07.413452 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:09.450251 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:09.450207 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:19.012371 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:19.012332 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:27.278252 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:27.278212 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:32.368824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:32.368786 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:35.291363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:35.291319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:35.390172 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:35.390125 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:35.750821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:35.750791 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:41.029012 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.028974 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:41.126991 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.126951 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:41.752161 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.752130 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:41.852290 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.852220 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:41.918546 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.918516 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw"] Jul 15 20:20:41.919090 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.919070 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="07bb68c9-fde9-44f1-bc56-684e8031e6cf" containerName="cleanup" Jul 15 20:20:41.919199 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.919094 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="07bb68c9-fde9-44f1-bc56-684e8031e6cf" containerName="cleanup" Jul 15 20:20:41.919272 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.919210 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="07bb68c9-fde9-44f1-bc56-684e8031e6cf" containerName="cleanup" Jul 15 20:20:41.922556 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.922534 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.925426 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.925407 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-c8d33b32-openshift-default-dockercfg-mp5rc\"" Jul 15 20:20:41.944160 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.944129 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw"] Jul 15 20:20:41.956407 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956364 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:41.956545 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956431 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6bk9\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-kube-api-access-r6bk9\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956545 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956501 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-credential-socket\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956642 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956553 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-data\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956696 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956683 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-certs\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956743 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956717 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-envoy\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956803 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956781 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-socket\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956863 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-podinfo\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956917 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956867 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istiod-ca-cert\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:41.956917 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:41.956901 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-token\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.031506 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.031397 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:42.057577 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057541 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-certs\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057583 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-envoy\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057616 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-socket\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057643 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-podinfo\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057659 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istiod-ca-cert\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057678 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-token\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057707 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6bk9\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-kube-api-access-r6bk9\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.057748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057731 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-credential-socket\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.058122 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.057765 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-data\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.058122 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.058018 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-certs\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.058122 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.058032 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-socket\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.058336 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.058205 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-credential-socket\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.058336 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.058264 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-data\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.058737 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.058718 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istiod-ca-cert\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.060713 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.060678 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-envoy\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.060899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.060882 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-podinfo\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.066739 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.066716 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-token\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.067031 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.067009 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6bk9\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-kube-api-access-r6bk9\") pod \"e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.237030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.236992 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:42.383184 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.383156 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw"] Jul 15 20:20:42.385094 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:20:42.385062 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40dc7e29_a07a_43c8_9cc5_e5a688906c69.slice/crio-1f317b0a47170a62dd63ab03f1ad77ff4bd211cf844764946567e40239b559b1 WatchSource:0}: Error finding container 1f317b0a47170a62dd63ab03f1ad77ff4bd211cf844764946567e40239b559b1: Status 404 returned error can't find the container with id 1f317b0a47170a62dd63ab03f1ad77ff4bd211cf844764946567e40239b559b1 Jul 15 20:20:42.387446 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.387401 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:20:42.387522 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.387506 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:20:42.387570 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.387554 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:20:42.979062 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.979024 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" event={"ID":"40dc7e29-a07a-43c8-9cc5-e5a688906c69","Type":"ContainerStarted","Data":"d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c"} Jul 15 20:20:42.979279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:42.979074 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" event={"ID":"40dc7e29-a07a-43c8-9cc5-e5a688906c69","Type":"ContainerStarted","Data":"1f317b0a47170a62dd63ab03f1ad77ff4bd211cf844764946567e40239b559b1"} Jul 15 20:20:43.002411 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.002359 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podStartSLOduration=2.002341166 podStartE2EDuration="2.002341166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:20:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:20:43.000409905 +0000 UTC m=+1590.671196456" watchObservedRunningTime="2026-07-15 20:20:43.002341166 +0000 UTC m=+1590.673127691" Jul 15 20:20:43.060960 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.060891 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:43.149074 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.149036 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:43.215835 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.215804 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:43.237299 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.237189 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:43.265162 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.265131 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:43.430995 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.430964 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:43.763165 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.763134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:43.864277 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.864226 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:43.984559 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.984534 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:43.985521 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:43.985491 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:45.852967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:45.852936 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:20:48.159562 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.159526 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9"] Jul 15 20:20:48.163598 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.163581 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:20:48.166033 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.166005 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:20:48.166157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.166115 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:20:48.171474 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.171451 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9"] Jul 15 20:20:48.224867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.224814 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41bc735e-fcb8-492d-b9ae-8c88ea4256c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9\" (UID: \"41bc735e-fcb8-492d-b9ae-8c88ea4256c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:20:48.326096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.326045 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41bc735e-fcb8-492d-b9ae-8c88ea4256c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9\" (UID: \"41bc735e-fcb8-492d-b9ae-8c88ea4256c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:20:48.327359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.327189 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41bc735e-fcb8-492d-b9ae-8c88ea4256c8-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9\" (UID: \"41bc735e-fcb8-492d-b9ae-8c88ea4256c8\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:20:48.476290 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.476226 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:20:48.670705 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:48.670674 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9"] Jul 15 20:20:48.672685 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:20:48.672657 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41bc735e_fcb8_492d_b9ae_8c88ea4256c8.slice/crio-ad6093968710faebb5189aa91e541aa9de7081f3041d4f384fc5d7767ba50da5 WatchSource:0}: Error finding container ad6093968710faebb5189aa91e541aa9de7081f3041d4f384fc5d7767ba50da5: Status 404 returned error can't find the container with id ad6093968710faebb5189aa91e541aa9de7081f3041d4f384fc5d7767ba50da5 Jul 15 20:20:49.016882 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:49.016846 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerStarted","Data":"ad6093968710faebb5189aa91e541aa9de7081f3041d4f384fc5d7767ba50da5"} Jul 15 20:20:50.029588 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:50.028272 2557 generic.go:356] "Generic (PLEG): container finished" podID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerID="8ef9b708bdec3fd671cb2c7a998bc7745bf1f8a588827dd795b339fb76783b73" exitCode=7 Jul 15 20:20:50.029588 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:50.028448 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerDied","Data":"8ef9b708bdec3fd671cb2c7a998bc7745bf1f8a588827dd795b339fb76783b73"} Jul 15 20:20:50.029588 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:50.028870 2557 scope.go:117] "RemoveContainer" containerID="8ef9b708bdec3fd671cb2c7a998bc7745bf1f8a588827dd795b339fb76783b73" Jul 15 20:20:51.039002 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:51.038966 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerStarted","Data":"93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733"} Jul 15 20:20:51.057754 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:51.057704 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" podStartSLOduration=3.057687765 podStartE2EDuration="3.057687765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:20:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:20:51.056967201 +0000 UTC m=+1598.727753727" watchObservedRunningTime="2026-07-15 20:20:51.057687765 +0000 UTC m=+1598.728474289" Jul 15 20:20:52.048563 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:52.048526 2557 generic.go:356] "Generic (PLEG): container finished" podID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerID="93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733" exitCode=7 Jul 15 20:20:52.049026 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:52.048592 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerDied","Data":"93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733"} Jul 15 20:20:52.049026 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:52.048723 2557 scope.go:117] "RemoveContainer" containerID="8ef9b708bdec3fd671cb2c7a998bc7745bf1f8a588827dd795b339fb76783b73" Jul 15 20:20:52.049026 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:52.048946 2557 scope.go:117] "RemoveContainer" containerID="93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733" Jul 15 20:20:52.049284 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:20:52.049232 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9_odh-ai-gateway-infra(41bc735e-fcb8-492d-b9ae-8c88ea4256c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" Jul 15 20:20:53.056709 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:53.056670 2557 scope.go:117] "RemoveContainer" containerID="93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733" Jul 15 20:20:53.057201 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:20:53.056964 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9_odh-ai-gateway-infra(41bc735e-fcb8-492d-b9ae-8c88ea4256c8)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" Jul 15 20:20:55.070249 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:55.070199 2557 generic.go:356] "Generic (PLEG): container finished" podID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerID="d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c" exitCode=0 Jul 15 20:20:55.070792 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:55.070279 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" event={"ID":"40dc7e29-a07a-43c8-9cc5-e5a688906c69","Type":"ContainerDied","Data":"d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c"} Jul 15 20:20:55.070861 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:55.070817 2557 scope.go:117] "RemoveContainer" containerID="d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c" Jul 15 20:20:55.072108 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:55.072078 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:20:55.072181 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:55.072165 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:20:55.072221 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:55.072207 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:20:56.078717 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:56.078676 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" event={"ID":"40dc7e29-a07a-43c8-9cc5-e5a688906c69","Type":"ContainerStarted","Data":"75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694"} Jul 15 20:20:56.238150 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:56.238114 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:56.239257 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:56.239207 2557 patch_prober.go:28] interesting pod/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 20:20:56.239435 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:56.239273 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 20:20:57.238410 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:57.238366 2557 patch_prober.go:28] interesting pod/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 20:20:57.238807 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:57.238443 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 20:20:58.237502 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:58.237470 2557 patch_prober.go:28] interesting pod/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 20:20:58.237686 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:58.237533 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 20:20:58.984938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:58.984908 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:20:59.238130 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:59.238034 2557 patch_prober.go:28] interesting pod/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 20:20:59.238130 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:20:59.238091 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 20:21:00.238107 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:00.238066 2557 patch_prober.go:28] interesting pod/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" start-of-body= Jul 15 20:21:00.238542 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:00.238132 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.43:15021/healthz/ready\": dial tcp 10.132.0.43:15021: connect: connection refused" Jul 15 20:21:01.241977 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:01.241947 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:21:01.242806 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:01.242784 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:21:07.780219 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:07.780127 2557 scope.go:117] "RemoveContainer" containerID="93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733" Jul 15 20:21:08.147414 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:08.147377 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerStarted","Data":"aa9aa5e20fd56d3775b6f5a94c53bbaab7b7d3a35720baacc1e6e88663c2051e"} Jul 15 20:21:09.154278 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:09.154215 2557 generic.go:356] "Generic (PLEG): container finished" podID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerID="aa9aa5e20fd56d3775b6f5a94c53bbaab7b7d3a35720baacc1e6e88663c2051e" exitCode=0 Jul 15 20:21:09.154790 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:09.154297 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerDied","Data":"aa9aa5e20fd56d3775b6f5a94c53bbaab7b7d3a35720baacc1e6e88663c2051e"} Jul 15 20:21:09.154790 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:09.154361 2557 scope.go:117] "RemoveContainer" containerID="93a739e1cfd3d85413cea76073c47e72f8ca6621a846f9ff4ab57fe3591e1733" Jul 15 20:21:09.180716 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:09.180677 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9"] Jul 15 20:21:10.311020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:10.310993 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:21:10.358994 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:10.358967 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41bc735e-fcb8-492d-b9ae-8c88ea4256c8-maas-api-service-ca\") pod \"41bc735e-fcb8-492d-b9ae-8c88ea4256c8\" (UID: \"41bc735e-fcb8-492d-b9ae-8c88ea4256c8\") " Jul 15 20:21:10.359336 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:10.359312 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41bc735e-fcb8-492d-b9ae-8c88ea4256c8-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41bc735e-fcb8-492d-b9ae-8c88ea4256c8" (UID: "41bc735e-fcb8-492d-b9ae-8c88ea4256c8"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:21:10.459940 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:10.459908 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41bc735e-fcb8-492d-b9ae-8c88ea4256c8-maas-api-service-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:21:11.167189 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:11.167106 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" Jul 15 20:21:11.167189 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:11.167146 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9" event={"ID":"41bc735e-fcb8-492d-b9ae-8c88ea4256c8","Type":"ContainerDied","Data":"ad6093968710faebb5189aa91e541aa9de7081f3041d4f384fc5d7767ba50da5"} Jul 15 20:21:11.167394 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:11.167193 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ad6093968710faebb5189aa91e541aa9de7081f3041d4f384fc5d7767ba50da5" Jul 15 20:21:11.200390 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:11.200360 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9"] Jul 15 20:21:11.206443 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:11.206421 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-c8d33b32-jvwv9"] Jul 15 20:21:12.785765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:12.785732 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" path="/var/lib/kubelet/pods/41bc735e-fcb8-492d-b9ae-8c88ea4256c8/volumes" Jul 15 20:21:58.674578 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:21:58.674548 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:03.316732 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:03.316691 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:09.426208 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:09.426170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:09.486829 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:09.486794 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:13.479299 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:13.479263 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw"] Jul 15 20:22:13.479724 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:13.479496 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" containerID="cri-o://75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694" gracePeriod=30 Jul 15 20:22:13.503671 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:13.503634 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:13.865460 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:13.865371 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:13.961255 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:13.961215 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:14.047716 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.047677 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk"] Jul 15 20:22:14.048226 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.048206 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:14.048366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.048229 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:14.048366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.048288 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:14.048366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.048298 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:14.048525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.048416 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:14.048525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.048433 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:14.051788 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.051765 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.055397 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.055374 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-7ac342b1-gw-openshift-default-dockercfg-zrqtq\"" Jul 15 20:22:14.079999 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.079975 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk"] Jul 15 20:22:14.120124 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.120055 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:14.190337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190305 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-envoy\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190522 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190350 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-data\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190522 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190416 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-podinfo\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190522 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190471 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74mtz\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-kube-api-access-74mtz\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190672 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190533 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-socket\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190672 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190580 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-token\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190672 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190604 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-credential-socket\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190822 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190689 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-certs\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.190822 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.190754 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istiod-ca-cert\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.291654 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.291622 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-socket\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.291845 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.291675 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-token\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.291845 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.291692 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-credential-socket\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.291845 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.291816 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-certs\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.291880 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istiod-ca-cert\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.291928 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-envoy\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292124 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292019 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-credential-socket\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292124 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292059 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-data\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292124 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292085 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-socket\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292124 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292123 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-podinfo\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292311 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74mtz\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-kube-api-access-74mtz\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292311 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292160 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-certs\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292435 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292358 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-data\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.292651 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.292599 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istiod-ca-cert\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.294281 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.294261 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-envoy\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.294676 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.294661 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-podinfo\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.300131 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.300112 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-token\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.300589 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.300568 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74mtz\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-kube-api-access-74mtz\") pod \"e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.363024 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.362995 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:14.511685 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:22:14.511630 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41200327_e1dc_4f6a_8c16_e2243dcf45d5.slice/crio-339963403ec275097b6582442a85600c3a79d9fd32ce6887361f1ae8cd4b8967 WatchSource:0}: Error finding container 339963403ec275097b6582442a85600c3a79d9fd32ce6887361f1ae8cd4b8967: Status 404 returned error can't find the container with id 339963403ec275097b6582442a85600c3a79d9fd32ce6887361f1ae8cd4b8967 Jul 15 20:22:14.514041 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.513992 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:22:14.514157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.514059 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk"] Jul 15 20:22:14.514157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.514090 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:22:14.514157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:14.514139 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:22:15.342702 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:15.342663 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:15.463835 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:15.463800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:15.521337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:15.521306 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" event={"ID":"41200327-e1dc-4f6a-8c16-e2243dcf45d5","Type":"ContainerStarted","Data":"94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba"} Jul 15 20:22:15.521720 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:15.521366 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" event={"ID":"41200327-e1dc-4f6a-8c16-e2243dcf45d5","Type":"ContainerStarted","Data":"339963403ec275097b6582442a85600c3a79d9fd32ce6887361f1ae8cd4b8967"} Jul 15 20:22:15.544704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:15.544642 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" podStartSLOduration=1.544625104 podStartE2EDuration="1.544625104s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:22:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:22:15.542687178 +0000 UTC m=+1683.213473706" watchObservedRunningTime="2026-07-15 20:22:15.544625104 +0000 UTC m=+1683.215411629" Jul 15 20:22:15.620569 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:15.620483 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:16.043544 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.043511 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:16.142786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.142753 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:16.245405 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.245363 2557 patch_prober.go:28] interesting pod/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:22:16.245570 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.245432 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:22:16.363450 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.363372 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:16.368561 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.368539 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:16.526795 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.526763 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:16.527627 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:16.527605 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:18.774963 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.774938 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:22:18.943092 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943059 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istiod-ca-cert\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943092 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943095 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-token\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943335 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943152 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-socket\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943335 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943180 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-podinfo\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943335 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943273 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-envoy\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943335 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943306 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-credential-socket\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943530 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943343 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r6bk9\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-kube-api-access-r6bk9\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943530 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943408 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-certs\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943530 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943441 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-data\") pod \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\" (UID: \"40dc7e29-a07a-43c8-9cc5-e5a688906c69\") " Jul 15 20:22:18.943530 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943486 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:22:18.943530 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943494 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:22:18.943788 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943635 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:22:18.943788 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943646 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:22:18.943897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943838 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:18.943897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943857 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:18.943897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943871 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:18.943897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.943883 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:18.944327 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.944308 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-data" (OuterVolumeSpecName: "istio-data") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:22:18.945818 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.945780 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:22:18.945818 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.945809 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-kube-api-access-r6bk9" (OuterVolumeSpecName: "kube-api-access-r6bk9") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "kube-api-access-r6bk9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:22:18.945999 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.945980 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:22:18.946056 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:18.946022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-token" (OuterVolumeSpecName: "istio-token") pod "40dc7e29-a07a-43c8-9cc5-e5a688906c69" (UID: "40dc7e29-a07a-43c8-9cc5-e5a688906c69"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:22:19.044789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.044687 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:19.044789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.044738 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:19.044789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.044752 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:19.044789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.044762 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/40dc7e29-a07a-43c8-9cc5-e5a688906c69-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:19.044789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.044774 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6bk9\" (UniqueName: \"kubernetes.io/projected/40dc7e29-a07a-43c8-9cc5-e5a688906c69-kube-api-access-r6bk9\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:19.544838 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.544805 2557 generic.go:356] "Generic (PLEG): container finished" podID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerID="75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694" exitCode=0 Jul 15 20:22:19.545020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.544846 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" Jul 15 20:22:19.545020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.544889 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" event={"ID":"40dc7e29-a07a-43c8-9cc5-e5a688906c69","Type":"ContainerDied","Data":"75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694"} Jul 15 20:22:19.545020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.544947 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw" event={"ID":"40dc7e29-a07a-43c8-9cc5-e5a688906c69","Type":"ContainerDied","Data":"1f317b0a47170a62dd63ab03f1ad77ff4bd211cf844764946567e40239b559b1"} Jul 15 20:22:19.545020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.544997 2557 scope.go:117] "RemoveContainer" containerID="75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694" Jul 15 20:22:19.557131 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.557111 2557 scope.go:117] "RemoveContainer" containerID="d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c" Jul 15 20:22:19.565812 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.565786 2557 scope.go:117] "RemoveContainer" containerID="75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694" Jul 15 20:22:19.566215 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:22:19.566100 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694\": container with ID starting with 75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694 not found: ID does not exist" containerID="75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694" Jul 15 20:22:19.566215 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.566133 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694"} err="failed to get container status \"75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694\": rpc error: code = NotFound desc = could not find container \"75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694\": container with ID starting with 75642ad2005bf8c6edd4c91f1cd77954c18b797ca7cb5020ae0246b47e5e4694 not found: ID does not exist" Jul 15 20:22:19.566215 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.566152 2557 scope.go:117] "RemoveContainer" containerID="d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c" Jul 15 20:22:19.566599 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:22:19.566558 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c\": container with ID starting with d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c not found: ID does not exist" containerID="d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c" Jul 15 20:22:19.566710 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.566590 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c"} err="failed to get container status \"d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c\": rpc error: code = NotFound desc = could not find container \"d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c\": container with ID starting with d6c360a341ecc7e520e93b55eb13b2d27d22231d15cf004f4a13d169af0ea79c not found: ID does not exist" Jul 15 20:22:19.569027 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.569007 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw"] Jul 15 20:22:19.575351 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.575331 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-c8d33b32-openshift-default-fb9b6c5f9-h9llw"] Jul 15 20:22:19.845312 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845214 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp"] Jul 15 20:22:19.845688 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845674 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" Jul 15 20:22:19.845732 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845689 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" Jul 15 20:22:19.845732 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845700 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" Jul 15 20:22:19.845732 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845706 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" Jul 15 20:22:19.845732 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845716 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:19.845732 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845723 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:19.845877 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845797 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" Jul 15 20:22:19.845877 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.845809 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41bc735e-fcb8-492d-b9ae-8c88ea4256c8" containerName="revoke-keys" Jul 15 20:22:19.850519 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.850500 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:19.853185 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.853158 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:22:19.853369 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.853210 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:22:19.856710 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.856676 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp"] Jul 15 20:22:19.952383 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:19.952352 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fedb42a3-21f5-44d2-9cc0-5717c485ff9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp\" (UID: \"fedb42a3-21f5-44d2-9cc0-5717c485ff9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:20.052964 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:20.052924 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fedb42a3-21f5-44d2-9cc0-5717c485ff9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp\" (UID: \"fedb42a3-21f5-44d2-9cc0-5717c485ff9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:20.053569 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:20.053551 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fedb42a3-21f5-44d2-9cc0-5717c485ff9b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp\" (UID: \"fedb42a3-21f5-44d2-9cc0-5717c485ff9b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:20.163552 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:20.163456 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:20.309127 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:20.309099 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp"] Jul 15 20:22:20.312281 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:22:20.312251 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfedb42a3_21f5_44d2_9cc0_5717c485ff9b.slice/crio-3b80186676a04be151be0ba70e1bf4beda49b4275c16027cf9ed7d7052f22140 WatchSource:0}: Error finding container 3b80186676a04be151be0ba70e1bf4beda49b4275c16027cf9ed7d7052f22140: Status 404 returned error can't find the container with id 3b80186676a04be151be0ba70e1bf4beda49b4275c16027cf9ed7d7052f22140 Jul 15 20:22:20.551027 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:20.550991 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" event={"ID":"fedb42a3-21f5-44d2-9cc0-5717c485ff9b","Type":"ContainerStarted","Data":"3b80186676a04be151be0ba70e1bf4beda49b4275c16027cf9ed7d7052f22140"} Jul 15 20:22:20.786803 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:20.786718 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" path="/var/lib/kubelet/pods/40dc7e29-a07a-43c8-9cc5-e5a688906c69/volumes" Jul 15 20:22:21.559453 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:21.559419 2557 generic.go:356] "Generic (PLEG): container finished" podID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerID="4a3fadd8e04be6e7d285fd054e2150872ad9dd5559c13c566e86939c195b2791" exitCode=7 Jul 15 20:22:21.559854 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:21.559505 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" event={"ID":"fedb42a3-21f5-44d2-9cc0-5717c485ff9b","Type":"ContainerDied","Data":"4a3fadd8e04be6e7d285fd054e2150872ad9dd5559c13c566e86939c195b2791"} Jul 15 20:22:21.559854 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:21.559686 2557 scope.go:117] "RemoveContainer" containerID="4a3fadd8e04be6e7d285fd054e2150872ad9dd5559c13c566e86939c195b2791" Jul 15 20:22:22.566807 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:22.566768 2557 generic.go:356] "Generic (PLEG): container finished" podID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerID="0b400d6249326724dcc7cd1f2403d49af5ee4cf78c0d88c5fe417400f5a276ab" exitCode=7 Jul 15 20:22:22.567207 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:22.566847 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" event={"ID":"fedb42a3-21f5-44d2-9cc0-5717c485ff9b","Type":"ContainerDied","Data":"0b400d6249326724dcc7cd1f2403d49af5ee4cf78c0d88c5fe417400f5a276ab"} Jul 15 20:22:22.567207 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:22.566902 2557 scope.go:117] "RemoveContainer" containerID="4a3fadd8e04be6e7d285fd054e2150872ad9dd5559c13c566e86939c195b2791" Jul 15 20:22:22.567207 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:22.567107 2557 scope.go:117] "RemoveContainer" containerID="0b400d6249326724dcc7cd1f2403d49af5ee4cf78c0d88c5fe417400f5a276ab" Jul 15 20:22:22.567423 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:22:22.567403 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp_odh-ai-gateway-infra(fedb42a3-21f5-44d2-9cc0-5717c485ff9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" Jul 15 20:22:23.574208 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:23.574181 2557 scope.go:117] "RemoveContainer" containerID="0b400d6249326724dcc7cd1f2403d49af5ee4cf78c0d88c5fe417400f5a276ab" Jul 15 20:22:23.574610 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:22:23.574450 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp_odh-ai-gateway-infra(fedb42a3-21f5-44d2-9cc0-5717c485ff9b)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" Jul 15 20:22:28.602587 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:28.602491 2557 generic.go:356] "Generic (PLEG): container finished" podID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerID="94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba" exitCode=0 Jul 15 20:22:28.602587 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:28.602565 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" event={"ID":"41200327-e1dc-4f6a-8c16-e2243dcf45d5","Type":"ContainerDied","Data":"94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba"} Jul 15 20:22:28.603038 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:28.602975 2557 scope.go:117] "RemoveContainer" containerID="94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba" Jul 15 20:22:28.604042 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:28.604011 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:22:28.604156 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:28.604081 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:22:28.604156 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:28.604112 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:22:29.609667 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:29.609629 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" event={"ID":"41200327-e1dc-4f6a-8c16-e2243dcf45d5","Type":"ContainerStarted","Data":"18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048"} Jul 15 20:22:30.224265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:30.222165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:30.323068 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:30.323023 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:30.363704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:30.363657 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:30.364979 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:30.364949 2557 patch_prober.go:28] interesting pod/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 20:22:30.365095 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:30.365011 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 20:22:31.363666 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:31.363636 2557 patch_prober.go:28] interesting pod/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 20:22:31.364076 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:31.363693 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 20:22:31.526824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:31.526792 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:32.363578 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:32.363539 2557 patch_prober.go:28] interesting pod/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 20:22:32.363768 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:32.363600 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 20:22:33.363629 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:33.363578 2557 patch_prober.go:28] interesting pod/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" start-of-body= Jul 15 20:22:33.363826 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:33.363652 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.45:15021/healthz/ready\": dial tcp 10.132.0.45:15021: connect: connection refused" Jul 15 20:22:34.199442 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:34.199392 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:22:34.367503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:34.367473 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:34.368494 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:34.368474 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:22:34.779995 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:34.779909 2557 scope.go:117] "RemoveContainer" containerID="0b400d6249326724dcc7cd1f2403d49af5ee4cf78c0d88c5fe417400f5a276ab" Jul 15 20:22:35.653651 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:35.653616 2557 generic.go:356] "Generic (PLEG): container finished" podID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerID="43bab748f22b0161ea102d975ef3f30e9ebc5a5322bbff6979010376844bb0b3" exitCode=0 Jul 15 20:22:35.654090 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:35.653692 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" event={"ID":"fedb42a3-21f5-44d2-9cc0-5717c485ff9b","Type":"ContainerDied","Data":"43bab748f22b0161ea102d975ef3f30e9ebc5a5322bbff6979010376844bb0b3"} Jul 15 20:22:35.654090 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:35.653770 2557 scope.go:117] "RemoveContainer" containerID="0b400d6249326724dcc7cd1f2403d49af5ee4cf78c0d88c5fe417400f5a276ab" Jul 15 20:22:36.688112 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:36.688062 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp"] Jul 15 20:22:36.808109 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:36.808085 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:36.926898 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:36.926870 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fedb42a3-21f5-44d2-9cc0-5717c485ff9b-maas-api-service-ca\") pod \"fedb42a3-21f5-44d2-9cc0-5717c485ff9b\" (UID: \"fedb42a3-21f5-44d2-9cc0-5717c485ff9b\") " Jul 15 20:22:36.927328 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:36.927307 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fedb42a3-21f5-44d2-9cc0-5717c485ff9b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fedb42a3-21f5-44d2-9cc0-5717c485ff9b" (UID: "fedb42a3-21f5-44d2-9cc0-5717c485ff9b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:22:37.028560 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:37.028528 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fedb42a3-21f5-44d2-9cc0-5717c485ff9b-maas-api-service-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:22:37.667569 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:37.667535 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" event={"ID":"fedb42a3-21f5-44d2-9cc0-5717c485ff9b","Type":"ContainerDied","Data":"3b80186676a04be151be0ba70e1bf4beda49b4275c16027cf9ed7d7052f22140"} Jul 15 20:22:37.667569 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:37.667557 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp" Jul 15 20:22:37.667786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:37.667585 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3b80186676a04be151be0ba70e1bf4beda49b4275c16027cf9ed7d7052f22140" Jul 15 20:22:37.700939 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:37.700906 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp"] Jul 15 20:22:37.706571 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:37.706546 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-7ac342b1-nfdpp"] Jul 15 20:22:38.786294 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:22:38.786226 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" path="/var/lib/kubelet/pods/fedb42a3-21f5-44d2-9cc0-5717c485ff9b/volumes" Jul 15 20:23:19.952312 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:19.952274 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk"] Jul 15 20:23:19.952822 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:19.952501 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" containerID="cri-o://18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048" gracePeriod=30 Jul 15 20:23:19.983870 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:19.983834 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:21.599469 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:21.599434 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:25.220758 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.220674 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:23:25.314069 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314038 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istiod-ca-cert\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314280 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314121 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-podinfo\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314280 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314159 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-socket\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314280 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314192 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-envoy\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314280 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314221 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-certs\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314283 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-token\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314340 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74mtz\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-kube-api-access-74mtz\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314427 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-data\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314433 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:23:25.314503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314453 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-credential-socket\") pod \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\" (UID: \"41200327-e1dc-4f6a-8c16-e2243dcf45d5\") " Jul 15 20:23:25.314730 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314501 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:23:25.314730 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314644 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:23:25.314863 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314826 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.314863 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314848 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.314863 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314860 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.315003 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.314878 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:23:25.315886 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.315846 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-data" (OuterVolumeSpecName: "istio-data") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:23:25.316901 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.316873 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-token" (OuterVolumeSpecName: "istio-token") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:23:25.317149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.317122 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-kube-api-access-74mtz" (OuterVolumeSpecName: "kube-api-access-74mtz") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "kube-api-access-74mtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:23:25.317329 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.317314 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:23:25.317394 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.317372 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "41200327-e1dc-4f6a-8c16-e2243dcf45d5" (UID: "41200327-e1dc-4f6a-8c16-e2243dcf45d5"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:23:25.416033 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.415999 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.416033 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.416027 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.416033 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.416038 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.416345 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.416046 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74mtz\" (UniqueName: \"kubernetes.io/projected/41200327-e1dc-4f6a-8c16-e2243dcf45d5-kube-api-access-74mtz\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.416345 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.416056 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.416345 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.416064 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/41200327-e1dc-4f6a-8c16-e2243dcf45d5-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:25.935874 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.935841 2557 generic.go:356] "Generic (PLEG): container finished" podID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerID="18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048" exitCode=0 Jul 15 20:23:25.936096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.935887 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" Jul 15 20:23:25.936096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.935932 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" event={"ID":"41200327-e1dc-4f6a-8c16-e2243dcf45d5","Type":"ContainerDied","Data":"18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048"} Jul 15 20:23:25.936096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.935996 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk" event={"ID":"41200327-e1dc-4f6a-8c16-e2243dcf45d5","Type":"ContainerDied","Data":"339963403ec275097b6582442a85600c3a79d9fd32ce6887361f1ae8cd4b8967"} Jul 15 20:23:25.936096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.936024 2557 scope.go:117] "RemoveContainer" containerID="18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048" Jul 15 20:23:25.946382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.946363 2557 scope.go:117] "RemoveContainer" containerID="94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba" Jul 15 20:23:25.955069 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.955049 2557 scope.go:117] "RemoveContainer" containerID="18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048" Jul 15 20:23:25.955378 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:23:25.955351 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048\": container with ID starting with 18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048 not found: ID does not exist" containerID="18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048" Jul 15 20:23:25.955431 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.955392 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048"} err="failed to get container status \"18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048\": rpc error: code = NotFound desc = could not find container \"18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048\": container with ID starting with 18401f718e46290d7ad7765d4403273684e0437eb5e5674d3835ea326ff1e048 not found: ID does not exist" Jul 15 20:23:25.955431 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.955415 2557 scope.go:117] "RemoveContainer" containerID="94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba" Jul 15 20:23:25.955635 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:23:25.955618 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba\": container with ID starting with 94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba not found: ID does not exist" containerID="94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba" Jul 15 20:23:25.955678 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.955641 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba"} err="failed to get container status \"94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba\": rpc error: code = NotFound desc = could not find container \"94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba\": container with ID starting with 94bb42d24c180298c418440aa365934d7a2fb4476b43407c4e3873d89861baba not found: ID does not exist" Jul 15 20:23:25.962624 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.962592 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk"] Jul 15 20:23:25.968912 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:25.968884 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-7ac342b1-gw-openshift-default-6b4c5676ff-jc4xk"] Jul 15 20:23:26.509786 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.509754 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:26.608934 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.608896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:26.712326 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712294 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl"] Jul 15 20:23:26.712854 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712838 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712856 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712873 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712879 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712904 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712912 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712920 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" Jul 15 20:23:26.712931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.712926 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" Jul 15 20:23:26.713228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.713001 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.713228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.713010 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" Jul 15 20:23:26.713228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.713018 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.713228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.713024 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" containerName="istio-proxy" Jul 15 20:23:26.713228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.713030 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:26.713228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.713037 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="40dc7e29-a07a-43c8-9cc5-e5a688906c69" containerName="istio-proxy" Jul 15 20:23:26.717789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.717763 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.720829 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.720801 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-dc00279d-openshift-default-dockercfg-72k42\"" Jul 15 20:23:26.732113 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.732085 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl"] Jul 15 20:23:26.754153 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.754118 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:26.787777 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.787699 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41200327-e1dc-4f6a-8c16-e2243dcf45d5" path="/var/lib/kubelet/pods/41200327-e1dc-4f6a-8c16-e2243dcf45d5/volumes" Jul 15 20:23:26.829734 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829696 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-envoy\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.829734 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829738 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-credential-socket\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830052 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829759 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-certs\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830052 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829858 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istiod-ca-cert\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830052 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829907 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-socket\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830052 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829935 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-data\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830052 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.829970 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-token\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830052 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.830032 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmqz2\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-kube-api-access-mmqz2\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.830401 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.830095 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-podinfo\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931004 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.930956 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-envoy\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931017 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-credential-socket\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931044 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-certs\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istiod-ca-cert\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931135 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-socket\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931168 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-data\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931529 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931329 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-token\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931529 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931389 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmqz2\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-kube-api-access-mmqz2\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931529 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931456 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-podinfo\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931523 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-certs\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931634 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-data\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931669 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-socket\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931874 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931857 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-credential-socket\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.931941 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.931899 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istiod-ca-cert\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.933526 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.933508 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-envoy\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.934050 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.934034 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-podinfo\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.945256 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.945207 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-token\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:26.945633 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:26.945609 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmqz2\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-kube-api-access-mmqz2\") pod \"e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:27.031947 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.031911 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:27.182739 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.182714 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl"] Jul 15 20:23:27.184255 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:23:27.184213 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd36e7e0_427d_42f2_b76c_d9e6bad9a3dd.slice/crio-fceb940c5111bc5780a4dad4ec467aafce4f116879ece32109328cc6897b53f0 WatchSource:0}: Error finding container fceb940c5111bc5780a4dad4ec467aafce4f116879ece32109328cc6897b53f0: Status 404 returned error can't find the container with id fceb940c5111bc5780a4dad4ec467aafce4f116879ece32109328cc6897b53f0 Jul 15 20:23:27.186401 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.186366 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:23:27.186501 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.186437 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:23:27.186501 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.186466 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:23:27.860876 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.860839 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:27.948661 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.948628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:27.952778 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.952744 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" event={"ID":"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd","Type":"ContainerStarted","Data":"8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d"} Jul 15 20:23:27.952959 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.952820 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" event={"ID":"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd","Type":"ContainerStarted","Data":"fceb940c5111bc5780a4dad4ec467aafce4f116879ece32109328cc6897b53f0"} Jul 15 20:23:27.983390 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:27.983339 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podStartSLOduration=1.983321999 podStartE2EDuration="1.983321999s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:23:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:23:27.98178049 +0000 UTC m=+1755.652567049" watchObservedRunningTime="2026-07-15 20:23:27.983321999 +0000 UTC m=+1755.654108564" Jul 15 20:23:28.023227 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.023177 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:28.033153 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.033126 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:28.038451 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.038428 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:28.544836 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.544800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:28.653534 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.653502 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:28.761432 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.761398 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:28.958118 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.958089 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:28.959099 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:28.959080 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:31.877104 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:31.877063 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:31.977364 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:31.977332 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:32.951632 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.951594 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq"] Jul 15 20:23:32.952057 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.952044 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:32.952057 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.952057 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="fedb42a3-21f5-44d2-9cc0-5717c485ff9b" containerName="revoke-keys" Jul 15 20:23:32.955608 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.955591 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:32.958688 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.958664 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:23:32.958819 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.958664 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:23:32.963159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.963132 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq"] Jul 15 20:23:32.986593 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:32.986566 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/684249a5-1145-4d4a-bf32-eea838a01ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq\" (UID: \"684249a5-1145-4d4a-bf32-eea838a01ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:33.088158 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:33.088114 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/684249a5-1145-4d4a-bf32-eea838a01ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq\" (UID: \"684249a5-1145-4d4a-bf32-eea838a01ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:33.088833 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:33.088814 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/684249a5-1145-4d4a-bf32-eea838a01ac4-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq\" (UID: \"684249a5-1145-4d4a-bf32-eea838a01ac4\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:33.267505 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:33.267419 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:33.469493 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:33.469463 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq"] Jul 15 20:23:33.472703 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:23:33.472672 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod684249a5_1145_4d4a_bf32_eea838a01ac4.slice/crio-b17031184a8f535d49c1af2719c070d61e6cceea19f2dc65d0c30d1d48e3bef6 WatchSource:0}: Error finding container b17031184a8f535d49c1af2719c070d61e6cceea19f2dc65d0c30d1d48e3bef6: Status 404 returned error can't find the container with id b17031184a8f535d49c1af2719c070d61e6cceea19f2dc65d0c30d1d48e3bef6 Jul 15 20:23:33.992779 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:33.992742 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" event={"ID":"684249a5-1145-4d4a-bf32-eea838a01ac4","Type":"ContainerStarted","Data":"f73e4dbb7e0c8566219678629d2f0f2dcfc2bb0e1a07023ae2aacdee98e1f8af"} Jul 15 20:23:33.993159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:33.992809 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" event={"ID":"684249a5-1145-4d4a-bf32-eea838a01ac4","Type":"ContainerStarted","Data":"b17031184a8f535d49c1af2719c070d61e6cceea19f2dc65d0c30d1d48e3bef6"} Jul 15 20:23:34.013203 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:34.013134 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" podStartSLOduration=2.013113228 podStartE2EDuration="2.013113228s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:23:34.009390402 +0000 UTC m=+1761.680176928" watchObservedRunningTime="2026-07-15 20:23:34.013113228 +0000 UTC m=+1761.683899754" Jul 15 20:23:34.999657 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:34.999621 2557 generic.go:356] "Generic (PLEG): container finished" podID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerID="f73e4dbb7e0c8566219678629d2f0f2dcfc2bb0e1a07023ae2aacdee98e1f8af" exitCode=7 Jul 15 20:23:35.000049 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:34.999699 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" event={"ID":"684249a5-1145-4d4a-bf32-eea838a01ac4","Type":"ContainerDied","Data":"f73e4dbb7e0c8566219678629d2f0f2dcfc2bb0e1a07023ae2aacdee98e1f8af"} Jul 15 20:23:35.000049 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:34.999854 2557 scope.go:117] "RemoveContainer" containerID="f73e4dbb7e0c8566219678629d2f0f2dcfc2bb0e1a07023ae2aacdee98e1f8af" Jul 15 20:23:35.300167 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:35.300131 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:36.007126 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:36.007097 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" event={"ID":"684249a5-1145-4d4a-bf32-eea838a01ac4","Type":"ContainerStarted","Data":"8ccd83e97654731e10cc244072c42ea30a925e381ba8bd8cca1bca2b4060957f"} Jul 15 20:23:37.020029 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:37.019318 2557 generic.go:356] "Generic (PLEG): container finished" podID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerID="8ccd83e97654731e10cc244072c42ea30a925e381ba8bd8cca1bca2b4060957f" exitCode=0 Jul 15 20:23:37.020029 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:37.019515 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" event={"ID":"684249a5-1145-4d4a-bf32-eea838a01ac4","Type":"ContainerDied","Data":"8ccd83e97654731e10cc244072c42ea30a925e381ba8bd8cca1bca2b4060957f"} Jul 15 20:23:37.020029 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:37.019573 2557 scope.go:117] "RemoveContainer" containerID="f73e4dbb7e0c8566219678629d2f0f2dcfc2bb0e1a07023ae2aacdee98e1f8af" Jul 15 20:23:38.182793 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:38.182764 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:38.260316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:38.260280 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/684249a5-1145-4d4a-bf32-eea838a01ac4-maas-api-service-ca\") pod \"684249a5-1145-4d4a-bf32-eea838a01ac4\" (UID: \"684249a5-1145-4d4a-bf32-eea838a01ac4\") " Jul 15 20:23:38.260675 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:38.260647 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/684249a5-1145-4d4a-bf32-eea838a01ac4-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "684249a5-1145-4d4a-bf32-eea838a01ac4" (UID: "684249a5-1145-4d4a-bf32-eea838a01ac4"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:23:38.362271 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:38.362168 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/684249a5-1145-4d4a-bf32-eea838a01ac4-maas-api-service-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:23:39.037833 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:39.037807 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" Jul 15 20:23:39.037985 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:39.037806 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq" event={"ID":"684249a5-1145-4d4a-bf32-eea838a01ac4","Type":"ContainerDied","Data":"b17031184a8f535d49c1af2719c070d61e6cceea19f2dc65d0c30d1d48e3bef6"} Jul 15 20:23:39.037985 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:39.037938 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b17031184a8f535d49c1af2719c070d61e6cceea19f2dc65d0c30d1d48e3bef6" Jul 15 20:23:40.045692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:40.045606 2557 generic.go:356] "Generic (PLEG): container finished" podID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerID="8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d" exitCode=0 Jul 15 20:23:40.046071 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:40.045687 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" event={"ID":"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd","Type":"ContainerDied","Data":"8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d"} Jul 15 20:23:40.046137 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:40.046122 2557 scope.go:117] "RemoveContainer" containerID="8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d" Jul 15 20:23:40.047183 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:40.047152 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:23:40.047319 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:40.047227 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:23:40.047319 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:40.047289 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:23:41.053215 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:41.053181 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" event={"ID":"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd","Type":"ContainerStarted","Data":"7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2"} Jul 15 20:23:42.033077 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:42.033030 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:42.034196 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:42.034173 2557 patch_prober.go:28] interesting pod/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 20:23:42.034349 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:42.034290 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 20:23:43.032985 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:43.032945 2557 patch_prober.go:28] interesting pod/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 20:23:43.033647 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:43.033609 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 20:23:43.598931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:43.598895 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:43.958513 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:43.958475 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:44.033142 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:44.033095 2557 patch_prober.go:28] interesting pod/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 20:23:44.033648 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:44.033176 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 20:23:45.033397 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:45.033356 2557 patch_prober.go:28] interesting pod/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body= Jul 15 20:23:45.033805 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:45.033425 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" Jul 15 20:23:46.036179 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:46.036149 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:46.037347 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:46.037325 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:23:48.104059 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:48.104010 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:49.821309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:49.821273 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:49.920014 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:49.919978 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:55.270450 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:55.270409 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:55.396460 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:55.396430 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:59.275492 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.275451 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl"] Jul 15 20:23:59.275973 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.275691 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" containerID="cri-o://7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2" gracePeriod=30 Jul 15 20:23:59.295129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.295098 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:59.705008 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.704971 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc"] Jul 15 20:23:59.705498 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.705483 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerName="revoke-keys" Jul 15 20:23:59.705548 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.705500 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerName="revoke-keys" Jul 15 20:23:59.705548 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.705511 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerName="revoke-keys" Jul 15 20:23:59.705548 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.705518 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerName="revoke-keys" Jul 15 20:23:59.705641 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.705580 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerName="revoke-keys" Jul 15 20:23:59.708768 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.708748 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.712283 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.712256 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-85451227-openshift-default-dockercfg-tcw2b\"" Jul 15 20:23:59.752151 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.752115 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc"] Jul 15 20:23:59.781920 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.781885 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:23:59.790767 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.790732 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-token\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.790944 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.790784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rxgfv\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-kube-api-access-rxgfv\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.790944 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.790874 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-data\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.790944 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.790901 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-certs\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.791080 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.790984 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-podinfo\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.791080 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.791034 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-socket\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.791080 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.791062 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istiod-ca-cert\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.791187 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.791105 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-envoy\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.791187 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.791155 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-credential-socket\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892469 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892433 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-token\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892469 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892474 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rxgfv\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-kube-api-access-rxgfv\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892512 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-data\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892529 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-certs\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892572 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-podinfo\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892602 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-socket\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892626 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istiod-ca-cert\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892670 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-envoy\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.892746 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.892723 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-credential-socket\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.893130 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.893015 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-data\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.893130 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.893035 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-socket\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.893130 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.893117 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-certs\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.893320 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.893193 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-credential-socket\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.893419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.893402 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istiod-ca-cert\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.895141 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.895123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-envoy\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.895208 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.895191 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-podinfo\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.908400 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.908371 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-token\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:23:59.910906 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:23:59.910880 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rxgfv\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-kube-api-access-rxgfv\") pod \"e2e-derive-85451227-openshift-default-974d68b59-gj8nc\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:00.020851 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:00.020769 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:00.177406 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:00.177368 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc"] Jul 15 20:24:00.181994 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:24:00.181966 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb91e6d1e_de69_4916_bd5d_9911c72ddaee.slice/crio-15bce5f92dcc74bddf5a35b8194f368a163d41fb3c55ccfae775fb965ab0dd8d WatchSource:0}: Error finding container 15bce5f92dcc74bddf5a35b8194f368a163d41fb3c55ccfae775fb965ab0dd8d: Status 404 returned error can't find the container with id 15bce5f92dcc74bddf5a35b8194f368a163d41fb3c55ccfae775fb965ab0dd8d Jul 15 20:24:00.184144 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:00.184112 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:24:00.184204 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:00.184176 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:24:00.184256 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:00.184206 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:24:00.998148 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:00.998118 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:01.038339 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.038298 2557 patch_prober.go:28] interesting pod/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:24:01.038500 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.038365 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:24:01.108511 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.108456 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:01.168901 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.168865 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" event={"ID":"b91e6d1e-de69-4916-bd5d-9911c72ddaee","Type":"ContainerStarted","Data":"b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639"} Jul 15 20:24:01.169101 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.168924 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" event={"ID":"b91e6d1e-de69-4916-bd5d-9911c72ddaee","Type":"ContainerStarted","Data":"15bce5f92dcc74bddf5a35b8194f368a163d41fb3c55ccfae775fb965ab0dd8d"} Jul 15 20:24:01.212534 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.212485 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podStartSLOduration=2.212469182 podStartE2EDuration="2.212469182s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:23:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:24:01.208299384 +0000 UTC m=+1788.879085919" watchObservedRunningTime="2026-07-15 20:24:01.212469182 +0000 UTC m=+1788.883255707" Jul 15 20:24:01.228089 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.228056 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:01.424603 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:01.424480 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:02.022049 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:02.022016 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:02.029042 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:02.029010 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:02.078348 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:02.078307 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:02.173423 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:02.173397 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:02.174919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:02.174899 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:02.175906 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:02.175887 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:04.538430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.538357 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:24:04.639601 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639568 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmqz2\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-kube-api-access-mmqz2\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.639772 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639633 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-podinfo\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.639772 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639674 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istiod-ca-cert\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.639772 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639705 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-envoy\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.639772 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639740 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-certs\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.639975 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639890 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-token\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.639975 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639946 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-data\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.640084 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.639984 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-credential-socket\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.640084 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.640041 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-socket\") pod \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\" (UID: \"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd\") " Jul 15 20:24:04.640216 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.640184 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:24:04.640658 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.640604 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.640760 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.640579 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:24:04.640829 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.640805 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:24:04.640890 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.640871 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:24:04.641037 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.641016 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-data" (OuterVolumeSpecName: "istio-data") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:24:04.642127 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.642103 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-token" (OuterVolumeSpecName: "istio-token") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:24:04.642444 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.642420 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:24:04.642686 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.642661 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:24:04.642795 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.642679 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-kube-api-access-mmqz2" (OuterVolumeSpecName: "kube-api-access-mmqz2") pod "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" (UID: "bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd"). InnerVolumeSpecName "kube-api-access-mmqz2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:24:04.741159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741121 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmqz2\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-kube-api-access-mmqz2\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741152 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741163 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741415 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741171 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741415 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741180 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741415 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741187 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741415 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741195 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:04.741415 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:04.741203 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:05.193059 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.193025 2557 generic.go:356] "Generic (PLEG): container finished" podID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerID="7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2" exitCode=0 Jul 15 20:24:05.193279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.193073 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" Jul 15 20:24:05.193279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.193106 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" event={"ID":"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd","Type":"ContainerDied","Data":"7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2"} Jul 15 20:24:05.193279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.193171 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl" event={"ID":"bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd","Type":"ContainerDied","Data":"fceb940c5111bc5780a4dad4ec467aafce4f116879ece32109328cc6897b53f0"} Jul 15 20:24:05.193279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.193205 2557 scope.go:117] "RemoveContainer" containerID="7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2" Jul 15 20:24:05.202701 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.202679 2557 scope.go:117] "RemoveContainer" containerID="8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d" Jul 15 20:24:05.211302 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.211281 2557 scope.go:117] "RemoveContainer" containerID="7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2" Jul 15 20:24:05.211591 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:24:05.211562 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2\": container with ID starting with 7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2 not found: ID does not exist" containerID="7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2" Jul 15 20:24:05.211643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.211590 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2"} err="failed to get container status \"7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2\": rpc error: code = NotFound desc = could not find container \"7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2\": container with ID starting with 7ad0de2006f6374996e9e34e963ba0bde963f85ea24c3700086dfeb98c3910f2 not found: ID does not exist" Jul 15 20:24:05.211643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.211610 2557 scope.go:117] "RemoveContainer" containerID="8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d" Jul 15 20:24:05.211848 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:24:05.211833 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d\": container with ID starting with 8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d not found: ID does not exist" containerID="8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d" Jul 15 20:24:05.211890 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.211852 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d"} err="failed to get container status \"8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d\": rpc error: code = NotFound desc = could not find container \"8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d\": container with ID starting with 8a3f1eff50ca1c2803ae489870a16a96b744d71840cd51ec063bcbc2768fcc6d not found: ID does not exist" Jul 15 20:24:05.226519 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.226494 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl"] Jul 15 20:24:05.231921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.231898 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-dc00279d-openshift-default-657cb4d78f-6fwhl"] Jul 15 20:24:05.284276 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284231 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh"] Jul 15 20:24:05.284707 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284695 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" Jul 15 20:24:05.284769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284709 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" Jul 15 20:24:05.284769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284726 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" Jul 15 20:24:05.284769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284740 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" Jul 15 20:24:05.284872 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284812 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" Jul 15 20:24:05.284872 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284821 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="684249a5-1145-4d4a-bf32-eea838a01ac4" containerName="revoke-keys" Jul 15 20:24:05.284872 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.284830 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" containerName="istio-proxy" Jul 15 20:24:05.289663 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.289646 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:05.293091 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.293069 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:24:05.293202 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.293069 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:24:05.297405 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.297380 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh"] Jul 15 20:24:05.346803 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.346769 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41a82839-b917-4521-8913-90c08222cac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-85451227-9lzrh\" (UID: \"41a82839-b917-4521-8913-90c08222cac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:05.447920 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.447831 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41a82839-b917-4521-8913-90c08222cac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-85451227-9lzrh\" (UID: \"41a82839-b917-4521-8913-90c08222cac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:05.448512 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.448492 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41a82839-b917-4521-8913-90c08222cac9-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-85451227-9lzrh\" (UID: \"41a82839-b917-4521-8913-90c08222cac9\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:05.600899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.600844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:05.760342 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.760307 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh"] Jul 15 20:24:05.763434 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:24:05.763405 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41a82839_b917_4521_8913_90c08222cac9.slice/crio-ca00265f84313b3d706e3a0722717faf5f6164a7f68e2ecd31031ad25ab70b73 WatchSource:0}: Error finding container ca00265f84313b3d706e3a0722717faf5f6164a7f68e2ecd31031ad25ab70b73: Status 404 returned error can't find the container with id ca00265f84313b3d706e3a0722717faf5f6164a7f68e2ecd31031ad25ab70b73 Jul 15 20:24:05.813502 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.813469 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:05.914363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:05.914332 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:06.201504 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:06.201464 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerStarted","Data":"b8f688ad0d8dbac0323928bf59e32ef96c7661b037029cb84c8dba33e3226a78"} Jul 15 20:24:06.201689 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:06.201523 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerStarted","Data":"ca00265f84313b3d706e3a0722717faf5f6164a7f68e2ecd31031ad25ab70b73"} Jul 15 20:24:06.233683 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:06.233623 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" podStartSLOduration=1.233605071 podStartE2EDuration="1.233605071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:24:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:24:06.231942411 +0000 UTC m=+1793.902728936" watchObservedRunningTime="2026-07-15 20:24:06.233605071 +0000 UTC m=+1793.904391595" Jul 15 20:24:06.787154 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:06.787075 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd" path="/var/lib/kubelet/pods/bd36e7e0-427d-42f2-b76c-d9e6bad9a3dd/volumes" Jul 15 20:24:07.208756 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:07.208714 2557 generic.go:356] "Generic (PLEG): container finished" podID="41a82839-b917-4521-8913-90c08222cac9" containerID="b8f688ad0d8dbac0323928bf59e32ef96c7661b037029cb84c8dba33e3226a78" exitCode=7 Jul 15 20:24:07.208941 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:07.208783 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerDied","Data":"b8f688ad0d8dbac0323928bf59e32ef96c7661b037029cb84c8dba33e3226a78"} Jul 15 20:24:07.208998 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:07.208974 2557 scope.go:117] "RemoveContainer" containerID="b8f688ad0d8dbac0323928bf59e32ef96c7661b037029cb84c8dba33e3226a78" Jul 15 20:24:08.223480 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:08.221881 2557 generic.go:356] "Generic (PLEG): container finished" podID="41a82839-b917-4521-8913-90c08222cac9" containerID="767e6eec2871da0d0ba64187bc37be059c077a4fd703496e85874772486e2907" exitCode=7 Jul 15 20:24:08.223480 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:08.221954 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerDied","Data":"767e6eec2871da0d0ba64187bc37be059c077a4fd703496e85874772486e2907"} Jul 15 20:24:08.223480 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:08.222052 2557 scope.go:117] "RemoveContainer" containerID="b8f688ad0d8dbac0323928bf59e32ef96c7661b037029cb84c8dba33e3226a78" Jul 15 20:24:08.223480 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:08.222539 2557 scope.go:117] "RemoveContainer" containerID="767e6eec2871da0d0ba64187bc37be059c077a4fd703496e85874772486e2907" Jul 15 20:24:08.223480 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:24:08.222787 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-85451227-9lzrh_odh-ai-gateway-infra(41a82839-b917-4521-8913-90c08222cac9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" podUID="41a82839-b917-4521-8913-90c08222cac9" Jul 15 20:24:09.234592 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:09.234563 2557 scope.go:117] "RemoveContainer" containerID="767e6eec2871da0d0ba64187bc37be059c077a4fd703496e85874772486e2907" Jul 15 20:24:09.235097 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:24:09.234782 2557 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"revoke-keys\" with CrashLoopBackOff: \"back-off 10s restarting failed container=revoke-keys pod=maas-api-revoke-keys-e2e-derive-85451227-9lzrh_odh-ai-gateway-infra(41a82839-b917-4521-8913-90c08222cac9)\"" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" podUID="41a82839-b917-4521-8913-90c08222cac9" Jul 15 20:24:09.254073 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:09.254033 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:15.149554 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.149517 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:24:15.271555 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.271517 2557 generic.go:356] "Generic (PLEG): container finished" podID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerID="b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639" exitCode=0 Jul 15 20:24:15.271743 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.271590 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" event={"ID":"b91e6d1e-de69-4916-bd5d-9911c72ddaee","Type":"ContainerDied","Data":"b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639"} Jul 15 20:24:15.272054 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.272038 2557 scope.go:117] "RemoveContainer" containerID="b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639" Jul 15 20:24:15.273081 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.273051 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:24:15.273181 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.273123 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:24:15.273181 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:15.273154 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:24:16.279283 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:16.279230 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" event={"ID":"b91e6d1e-de69-4916-bd5d-9911c72ddaee","Type":"ContainerStarted","Data":"459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e"} Jul 15 20:24:17.021772 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:17.021562 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:17.023302 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:17.022899 2557 patch_prober.go:28] interesting pod/e2e-derive-85451227-openshift-default-974d68b59-gj8nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 20:24:17.023302 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:17.022977 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 20:24:17.175205 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:17.175169 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:18.022056 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:18.022012 2557 patch_prober.go:28] interesting pod/e2e-derive-85451227-openshift-default-974d68b59-gj8nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 20:24:18.022706 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:18.022669 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 20:24:19.022143 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:19.022107 2557 patch_prober.go:28] interesting pod/e2e-derive-85451227-openshift-default-974d68b59-gj8nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 20:24:19.022764 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:19.022727 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 20:24:20.021807 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:20.021769 2557 patch_prober.go:28] interesting pod/e2e-derive-85451227-openshift-default-974d68b59-gj8nc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" start-of-body= Jul 15 20:24:20.022121 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:20.022090 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.49:15021/healthz/ready\": dial tcp 10.132.0.49:15021: connect: connection refused" Jul 15 20:24:20.780161 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:20.780129 2557 scope.go:117] "RemoveContainer" containerID="767e6eec2871da0d0ba64187bc37be059c077a4fd703496e85874772486e2907" Jul 15 20:24:21.025605 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:21.025574 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:21.026657 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:21.026635 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:24:21.318210 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:21.318171 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerStarted","Data":"4ba52b0a9fddb98c04645a854cabd3d011cbc0e9adc8efa50ba17bb542395faa"} Jul 15 20:24:22.325891 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:22.325857 2557 generic.go:356] "Generic (PLEG): container finished" podID="41a82839-b917-4521-8913-90c08222cac9" containerID="4ba52b0a9fddb98c04645a854cabd3d011cbc0e9adc8efa50ba17bb542395faa" exitCode=0 Jul 15 20:24:22.326373 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:22.325941 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerDied","Data":"4ba52b0a9fddb98c04645a854cabd3d011cbc0e9adc8efa50ba17bb542395faa"} Jul 15 20:24:22.326373 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:22.326019 2557 scope.go:117] "RemoveContainer" containerID="767e6eec2871da0d0ba64187bc37be059c077a4fd703496e85874772486e2907" Jul 15 20:24:22.355892 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:22.355858 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh"] Jul 15 20:24:23.478964 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:23.478939 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:23.645645 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:23.645538 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41a82839-b917-4521-8913-90c08222cac9-maas-api-service-ca\") pod \"41a82839-b917-4521-8913-90c08222cac9\" (UID: \"41a82839-b917-4521-8913-90c08222cac9\") " Jul 15 20:24:23.645934 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:23.645911 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/41a82839-b917-4521-8913-90c08222cac9-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "41a82839-b917-4521-8913-90c08222cac9" (UID: "41a82839-b917-4521-8913-90c08222cac9"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:24:23.746689 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:23.746646 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/41a82839-b917-4521-8913-90c08222cac9-maas-api-service-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:24:24.339291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:24.339232 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" Jul 15 20:24:24.339473 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:24.339255 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh" event={"ID":"41a82839-b917-4521-8913-90c08222cac9","Type":"ContainerDied","Data":"ca00265f84313b3d706e3a0722717faf5f6164a7f68e2ecd31031ad25ab70b73"} Jul 15 20:24:24.339473 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:24.339386 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca00265f84313b3d706e3a0722717faf5f6164a7f68e2ecd31031ad25ab70b73" Jul 15 20:24:24.381554 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:24.381519 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh"] Jul 15 20:24:24.387294 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:24.387229 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-85451227-9lzrh"] Jul 15 20:24:24.786028 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:24:24.785996 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="41a82839-b917-4521-8913-90c08222cac9" path="/var/lib/kubelet/pods/41a82839-b917-4521-8913-90c08222cac9/volumes" Jul 15 20:25:06.125459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:06.125426 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:10.450423 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:10.450383 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:16.764796 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:16.764763 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:16.863103 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:16.863070 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:20.621890 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:20.621851 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc"] Jul 15 20:25:20.622374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:20.622057 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" containerID="cri-o://459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e" gracePeriod=30 Jul 15 20:25:20.640617 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:20.640585 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:21.028630 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:21.028585 2557 patch_prober.go:28] interesting pod/e2e-derive-85451227-openshift-default-974d68b59-gj8nc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:25:21.028813 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:21.028660 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:25:21.302043 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:21.301956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:21.426885 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:21.426841 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:21.519118 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:21.519086 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:22.989624 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:22.989591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:25.883890 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.883828 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:25:25.967747 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967715 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-credential-socket\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.967942 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967772 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istiod-ca-cert\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.967942 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967800 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-data\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.967942 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967822 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-podinfo\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.967942 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967847 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-socket\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.967942 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967890 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-certs\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.968212 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.967955 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rxgfv\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-kube-api-access-rxgfv\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.968212 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968020 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-token\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.968212 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968058 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-envoy\") pod \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\" (UID: \"b91e6d1e-de69-4916-bd5d-9911c72ddaee\") " Jul 15 20:25:25.968212 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968118 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:25:25.968447 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968273 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:25:25.968548 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968520 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:25.968616 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968560 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:25.968673 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968611 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:25:25.968880 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.968840 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:25:25.969519 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.969471 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-data" (OuterVolumeSpecName: "istio-data") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:25:25.970833 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.970803 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:25:25.971174 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.971153 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-token" (OuterVolumeSpecName: "istio-token") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:25:25.971309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.971226 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-kube-api-access-rxgfv" (OuterVolumeSpecName: "kube-api-access-rxgfv") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "kube-api-access-rxgfv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:25:25.971460 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:25.971441 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b91e6d1e-de69-4916-bd5d-9911c72ddaee" (UID: "b91e6d1e-de69-4916-bd5d-9911c72ddaee"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:25:26.069142 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069104 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.069142 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069136 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.069142 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069147 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.069419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069158 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.069419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069170 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b91e6d1e-de69-4916-bd5d-9911c72ddaee-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.069419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069179 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b91e6d1e-de69-4916-bd5d-9911c72ddaee-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.069419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.069189 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rxgfv\" (UniqueName: \"kubernetes.io/projected/b91e6d1e-de69-4916-bd5d-9911c72ddaee-kube-api-access-rxgfv\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:25:26.687901 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.687866 2557 generic.go:356] "Generic (PLEG): container finished" podID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerID="459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e" exitCode=0 Jul 15 20:25:26.688097 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.687911 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" Jul 15 20:25:26.688097 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.687956 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" event={"ID":"b91e6d1e-de69-4916-bd5d-9911c72ddaee","Type":"ContainerDied","Data":"459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e"} Jul 15 20:25:26.688097 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.688014 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc" event={"ID":"b91e6d1e-de69-4916-bd5d-9911c72ddaee","Type":"ContainerDied","Data":"15bce5f92dcc74bddf5a35b8194f368a163d41fb3c55ccfae775fb965ab0dd8d"} Jul 15 20:25:26.688097 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.688037 2557 scope.go:117] "RemoveContainer" containerID="459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e" Jul 15 20:25:26.698219 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.698200 2557 scope.go:117] "RemoveContainer" containerID="b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639" Jul 15 20:25:26.706567 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.706550 2557 scope.go:117] "RemoveContainer" containerID="459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e" Jul 15 20:25:26.706814 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:25:26.706797 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e\": container with ID starting with 459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e not found: ID does not exist" containerID="459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e" Jul 15 20:25:26.706879 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.706821 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e"} err="failed to get container status \"459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e\": rpc error: code = NotFound desc = could not find container \"459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e\": container with ID starting with 459950d0d5c2f7e0900b3aee842be145237462af46f27a26022e907148eb881e not found: ID does not exist" Jul 15 20:25:26.706879 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.706838 2557 scope.go:117] "RemoveContainer" containerID="b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639" Jul 15 20:25:26.707054 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:25:26.707039 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639\": container with ID starting with b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639 not found: ID does not exist" containerID="b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639" Jul 15 20:25:26.707095 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.707058 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639"} err="failed to get container status \"b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639\": rpc error: code = NotFound desc = could not find container \"b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639\": container with ID starting with b64c2f25f7fb6f800eaa5d9c61f99e9cdd008ff1ab57ab7f28360a42cb16a639 not found: ID does not exist" Jul 15 20:25:26.729592 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.729568 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc"] Jul 15 20:25:26.735704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.735682 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-85451227-openshift-default-974d68b59-gj8nc"] Jul 15 20:25:26.785670 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:26.785642 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" path="/var/lib/kubelet/pods/b91e6d1e-de69-4916-bd5d-9911c72ddaee/volumes" Jul 15 20:25:27.358811 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:27.358783 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:29.554973 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:29.554934 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:29.665055 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:29.665018 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:29.758291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:29.758255 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:29.860289 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:29.860165 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:30.275662 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:30.275627 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:25:56.284614 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:25:56.284576 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:02.679573 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:02.679534 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:02.849359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:02.849319 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:02.978726 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:02.978638 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:18.799467 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:18.799435 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:25.039896 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:25.039860 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:25.111821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:25.111790 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:25.234624 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:25.234592 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:35.766886 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:35.766843 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:36.560023 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:36.559960 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:42.860606 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:42.860569 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:42.929394 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:42.929359 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:43.012673 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:43.012634 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:43.105627 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:43.105596 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:43.201940 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:43.201902 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:43.900727 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:43.900692 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:43.999270 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:43.999210 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:44.070848 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:44.070811 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:45.741207 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:45.741170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:48.971658 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:48.971626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:52.040158 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:52.040117 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:52.103231 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:52.103200 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:52.711087 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:52.711060 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:58.930335 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:58.930299 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:59.250843 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.250754 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:59.313349 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.313316 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:59.483357 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483318 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb"] Jul 15 20:26:59.483846 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483829 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483849 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483857 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483863 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483883 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483889 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483898 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" Jul 15 20:26:59.483915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483903 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" Jul 15 20:26:59.484168 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483981 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.484168 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.483994 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.484168 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.484006 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" Jul 15 20:26:59.484168 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.484012 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:26:59.484168 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.484018 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="b91e6d1e-de69-4916-bd5d-9911c72ddaee" containerName="istio-proxy" Jul 15 20:26:59.487325 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.487309 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.490079 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.490060 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-b8866fb3-openshift-default-dockercfg-gwdz8\"" Jul 15 20:26:59.492829 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.492794 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-socket\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.492958 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.492869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5712a258-1578-4fd8-a2bb-e0c1ee709275-istiod-ca-cert\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.492958 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.492923 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-token\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.493070 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.493024 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-certs\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.493070 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.493060 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-data\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.493149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.493090 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5qt9z\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-kube-api-access-5qt9z\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.493149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.493135 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-envoy\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.493225 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.493164 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-podinfo\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.493225 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.493194 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-credential-socket\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.519452 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.519384 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb"] Jul 15 20:26:59.560160 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.560125 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:59.594373 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594341 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-envoy\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594556 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594384 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-podinfo\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594556 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594425 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-credential-socket\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594556 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594462 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-socket\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594556 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594517 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5712a258-1578-4fd8-a2bb-e0c1ee709275-istiod-ca-cert\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594569 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-token\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594647 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-certs\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594672 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-data\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594697 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5qt9z\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-kube-api-access-5qt9z\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.594987 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.594888 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-credential-socket\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.595143 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.595110 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-certs\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.595143 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.595136 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-data\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.595390 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.595217 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-socket\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.595390 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.595285 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5712a258-1578-4fd8-a2bb-e0c1ee709275-istiod-ca-cert\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.596973 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.596945 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-envoy\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.597133 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.597115 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-podinfo\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.603198 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.603180 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-token\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.607721 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.607699 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5qt9z\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-kube-api-access-5qt9z\") pod \"e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.797328 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.797213 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:26:59.966760 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.966724 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb"] Jul 15 20:26:59.968727 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:26:59.968703 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5712a258_1578_4fd8_a2bb_e0c1ee709275.slice/crio-73c247437376fa6a209efdcb7a5afa895f80147b1efbfa4babf205bbbaad05f9 WatchSource:0}: Error finding container 73c247437376fa6a209efdcb7a5afa895f80147b1efbfa4babf205bbbaad05f9: Status 404 returned error can't find the container with id 73c247437376fa6a209efdcb7a5afa895f80147b1efbfa4babf205bbbaad05f9 Jul 15 20:26:59.970739 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.970711 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:26:59.970951 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.970920 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:26:59.970997 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.970988 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:26:59.971035 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:26:59.971016 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:27:00.210601 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:00.210555 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" event={"ID":"5712a258-1578-4fd8-a2bb-e0c1ee709275","Type":"ContainerStarted","Data":"c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33"} Jul 15 20:27:00.210802 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:00.210629 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" event={"ID":"5712a258-1578-4fd8-a2bb-e0c1ee709275","Type":"ContainerStarted","Data":"73c247437376fa6a209efdcb7a5afa895f80147b1efbfa4babf205bbbaad05f9"} Jul 15 20:27:00.242006 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:00.241503 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podStartSLOduration=1.241481595 podStartE2EDuration="1.241481595s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:26:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:27:00.239449338 +0000 UTC m=+1967.910235865" watchObservedRunningTime="2026-07-15 20:27:00.241481595 +0000 UTC m=+1967.912268121" Jul 15 20:27:00.797613 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:00.797585 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:01.803053 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:01.803021 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:02.222670 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:02.222642 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:02.223682 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:02.223663 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:02.384576 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:02.384546 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:02.491892 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:02.491801 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:02.622039 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:02.622000 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:02.948397 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:02.948362 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:03.468716 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:03.468688 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:03.586738 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:03.586703 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:06.813205 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:06.813161 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:07.463708 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:07.463678 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:07.559093 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:07.559059 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:07.906456 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:07.906375 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:11.594161 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:11.594123 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:11.697598 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:11.697565 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:12.967433 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:12.967401 2557 scope.go:117] "RemoveContainer" containerID="aa9aa5e20fd56d3775b6f5a94c53bbaab7b7d3a35720baacc1e6e88663c2051e" Jul 15 20:27:14.447694 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:14.447658 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:14.555116 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:14.555082 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:17.942893 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:17.941359 2557 patch_prober.go:28] interesting pod/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:58778->10.132.0.51:15021: read: connection reset by peer" start-of-body= Jul 15 20:27:17.942893 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:17.941429 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": read tcp 10.132.0.2:58778->10.132.0.51:15021: read: connection reset by peer" Jul 15 20:27:18.321567 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:18.321480 2557 generic.go:356] "Generic (PLEG): container finished" podID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerID="c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33" exitCode=0 Jul 15 20:27:18.321567 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:18.321552 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" event={"ID":"5712a258-1578-4fd8-a2bb-e0c1ee709275","Type":"ContainerDied","Data":"c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33"} Jul 15 20:27:18.321990 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:18.321977 2557 scope.go:117] "RemoveContainer" containerID="c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33" Jul 15 20:27:18.323021 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:18.322995 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:27:18.323075 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:18.323063 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:27:18.323119 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:18.323099 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:27:19.328609 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:19.328575 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" event={"ID":"5712a258-1578-4fd8-a2bb-e0c1ee709275","Type":"ContainerStarted","Data":"92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085"} Jul 15 20:27:19.798018 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:19.797974 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:19.798205 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:19.798187 2557 patch_prober.go:28] interesting pod/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 20:27:19.798300 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:19.798265 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 20:27:20.798262 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:20.798207 2557 patch_prober.go:28] interesting pod/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 20:27:20.798635 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:20.798285 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 20:27:21.798597 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:21.798561 2557 patch_prober.go:28] interesting pod/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 20:27:21.798997 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:21.798615 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 20:27:22.798459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:22.798421 2557 patch_prober.go:28] interesting pod/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" start-of-body= Jul 15 20:27:22.798665 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:22.798485 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.51:15021/healthz/ready\": dial tcp 10.132.0.51:15021: connect: connection refused" Jul 15 20:27:23.801710 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:23.801675 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:23.802113 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:23.802001 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:23.802738 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:23.802718 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:25.437959 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:25.437928 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:29.959493 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:29.959448 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:36.448987 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:36.448953 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:36.545204 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:36.545146 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:40.669460 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:40.669422 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb"] Jul 15 20:27:40.670015 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:40.669625 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" containerID="cri-o://92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085" gracePeriod=30 Jul 15 20:27:40.697933 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:40.697904 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:40.911642 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:40.911601 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:41.052707 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:41.052670 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:41.147748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:41.147709 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:41.312171 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:41.312073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:45.935087 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:45.935023 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:46.064663 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064626 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-certs\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.064868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064684 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-envoy\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.064868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064702 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-data\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.064868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064730 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-socket\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.064868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064768 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-token\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.064868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064837 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-podinfo\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.064868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064852 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-credential-socket\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.065182 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064954 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:27:46.065182 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065147 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:27:46.065337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.064895 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5qt9z\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-kube-api-access-5qt9z\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.065337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065221 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:27:46.065337 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065280 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5712a258-1578-4fd8-a2bb-e0c1ee709275-istiod-ca-cert\") pod \"5712a258-1578-4fd8-a2bb-e0c1ee709275\" (UID: \"5712a258-1578-4fd8-a2bb-e0c1ee709275\") " Jul 15 20:27:46.065619 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065568 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5712a258-1578-4fd8-a2bb-e0c1ee709275-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:27:46.065619 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065567 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-data" (OuterVolumeSpecName: "istio-data") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:27:46.065938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065796 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.065938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065853 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5712a258-1578-4fd8-a2bb-e0c1ee709275-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.065938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065867 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.065938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065879 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.065938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.065891 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.067309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.067279 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:27:46.067482 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.067462 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-token" (OuterVolumeSpecName: "istio-token") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:27:46.067747 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.067726 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:27:46.067747 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.067739 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-kube-api-access-5qt9z" (OuterVolumeSpecName: "kube-api-access-5qt9z") pod "5712a258-1578-4fd8-a2bb-e0c1ee709275" (UID: "5712a258-1578-4fd8-a2bb-e0c1ee709275"). InnerVolumeSpecName "kube-api-access-5qt9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:27:46.167437 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.167397 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.167437 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.167427 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.167437 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.167436 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5712a258-1578-4fd8-a2bb-e0c1ee709275-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.167437 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.167444 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5qt9z\" (UniqueName: \"kubernetes.io/projected/5712a258-1578-4fd8-a2bb-e0c1ee709275-kube-api-access-5qt9z\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:27:46.483154 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.483115 2557 generic.go:356] "Generic (PLEG): container finished" podID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerID="92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085" exitCode=0 Jul 15 20:27:46.483374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.483172 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" Jul 15 20:27:46.483374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.483168 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" event={"ID":"5712a258-1578-4fd8-a2bb-e0c1ee709275","Type":"ContainerDied","Data":"92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085"} Jul 15 20:27:46.483374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.483302 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb" event={"ID":"5712a258-1578-4fd8-a2bb-e0c1ee709275","Type":"ContainerDied","Data":"73c247437376fa6a209efdcb7a5afa895f80147b1efbfa4babf205bbbaad05f9"} Jul 15 20:27:46.483374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.483334 2557 scope.go:117] "RemoveContainer" containerID="92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085" Jul 15 20:27:46.493633 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.493613 2557 scope.go:117] "RemoveContainer" containerID="c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33" Jul 15 20:27:46.502302 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.502284 2557 scope.go:117] "RemoveContainer" containerID="92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085" Jul 15 20:27:46.502568 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:27:46.502550 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085\": container with ID starting with 92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085 not found: ID does not exist" containerID="92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085" Jul 15 20:27:46.502619 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.502577 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085"} err="failed to get container status \"92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085\": rpc error: code = NotFound desc = could not find container \"92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085\": container with ID starting with 92c11bda58975084a6bd4b953170692629beb5e432f20a983e2cee38fdb94085 not found: ID does not exist" Jul 15 20:27:46.502619 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.502596 2557 scope.go:117] "RemoveContainer" containerID="c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33" Jul 15 20:27:46.502867 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:27:46.502844 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33\": container with ID starting with c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33 not found: ID does not exist" containerID="c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33" Jul 15 20:27:46.502959 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.502871 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33"} err="failed to get container status \"c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33\": rpc error: code = NotFound desc = could not find container \"c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33\": container with ID starting with c4f14c8ab6740bda4ed0a3e39d27b3f498511bd1cb0bdef3c6d6ab046978de33 not found: ID does not exist" Jul 15 20:27:46.514690 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.514667 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb"] Jul 15 20:27:46.527560 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.527538 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-b8866fb3-openshift-default-8b4cbb9d7-c6sjb"] Jul 15 20:27:46.786673 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:46.786577 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" path="/var/lib/kubelet/pods/5712a258-1578-4fd8-a2bb-e0c1ee709275/volumes" Jul 15 20:27:47.971591 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:47.971555 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:48.079990 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:48.079952 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:48.171456 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:48.171420 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:48.404438 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:48.404354 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:54.558437 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:54.558406 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:27:54.662079 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:27:54.662045 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:01.443348 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.443293 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:01.741979 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.741871 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g"] Jul 15 20:28:01.742889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.742862 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" Jul 15 20:28:01.743134 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743102 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" Jul 15 20:28:01.743282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743142 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:28:01.743282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743153 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="41a82839-b917-4521-8913-90c08222cac9" containerName="revoke-keys" Jul 15 20:28:01.743282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743224 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" Jul 15 20:28:01.743282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743249 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" Jul 15 20:28:01.743493 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743471 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" Jul 15 20:28:01.743550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.743494 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5712a258-1578-4fd8-a2bb-e0c1ee709275" containerName="istio-proxy" Jul 15 20:28:01.748856 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.748826 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.751856 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.751836 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-83c1a2fd-openshift-default-dockercfg-svtll\"" Jul 15 20:28:01.759648 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.759624 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g"] Jul 15 20:28:01.811467 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.811426 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:01.924341 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924305 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-certs\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924519 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924460 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-envoy\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924565 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924548 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-socket\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924636 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924614 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-credential-socket\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924678 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924652 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxhw4\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-kube-api-access-qxhw4\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924736 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924721 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e625929b-76dd-4fd7-91f7-00719062180f-istio-podinfo\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924801 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924788 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-data\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924838 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924819 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-istio-token\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:01.924838 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:01.924834 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e625929b-76dd-4fd7-91f7-00719062180f-istiod-ca-cert\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.025891 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025799 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-envoy\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.025891 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025863 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-socket\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.025891 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025886 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-credential-socket\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxhw4\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-kube-api-access-qxhw4\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025937 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e625929b-76dd-4fd7-91f7-00719062180f-istio-podinfo\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025971 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-data\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.025997 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-istio-token\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026022 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e625929b-76dd-4fd7-91f7-00719062180f-istiod-ca-cert\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026051 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-certs\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026531 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026451 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-socket\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026531 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026473 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-data\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026531 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026499 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-certs\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026696 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026566 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-credential-socket\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.026994 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.026968 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e625929b-76dd-4fd7-91f7-00719062180f-istiod-ca-cert\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.028289 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.028271 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-envoy\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.028643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.028626 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e625929b-76dd-4fd7-91f7-00719062180f-istio-podinfo\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.035492 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.035466 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:02.043060 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.043031 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-istio-token\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.043285 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.043254 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxhw4\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-kube-api-access-qxhw4\") pod \"e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.061160 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.061131 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:02.210199 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.210160 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g"] Jul 15 20:28:02.216672 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.216633 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:28:02.216776 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.216708 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:28:02.216776 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.216737 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:28:02.575108 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.575074 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" event={"ID":"e625929b-76dd-4fd7-91f7-00719062180f","Type":"ContainerStarted","Data":"94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b"} Jul 15 20:28:02.575540 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.575126 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" event={"ID":"e625929b-76dd-4fd7-91f7-00719062180f","Type":"ContainerStarted","Data":"30ed6ac0aa69167966bafa54117d856552ff8afc62d6b9fbc885959972d0f9a8"} Jul 15 20:28:02.600847 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:02.600784 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podStartSLOduration=1.600767863 podStartE2EDuration="1.600767863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:28:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:28:02.598970879 +0000 UTC m=+2030.269757404" watchObservedRunningTime="2026-07-15 20:28:02.600767863 +0000 UTC m=+2030.271554365" Jul 15 20:28:03.062136 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:03.062098 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:03.067065 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:03.067039 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:03.581666 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:03.581638 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:03.582822 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:03.582803 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:04.610219 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:04.610183 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:04.705885 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:04.705848 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:04.842911 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:04.842877 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:04.980029 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:04.979996 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:05.329187 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:05.329094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:05.498096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:05.498060 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:09.071612 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:09.071574 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:16.941008 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:16.940972 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:17.672665 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:17.672551 2557 generic.go:356] "Generic (PLEG): container finished" podID="e625929b-76dd-4fd7-91f7-00719062180f" containerID="94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b" exitCode=0 Jul 15 20:28:17.672665 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:17.672616 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" event={"ID":"e625929b-76dd-4fd7-91f7-00719062180f","Type":"ContainerDied","Data":"94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b"} Jul 15 20:28:17.673188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:17.673171 2557 scope.go:117] "RemoveContainer" containerID="94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b" Jul 15 20:28:17.674399 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:17.674365 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:28:17.674660 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:17.674455 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:28:17.674660 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:17.674493 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:28:18.582482 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:18.582447 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:18.680367 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:18.680333 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" event={"ID":"e625929b-76dd-4fd7-91f7-00719062180f","Type":"ContainerStarted","Data":"ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076"} Jul 15 20:28:19.061740 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:19.061697 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:19.063094 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:19.063066 2557 patch_prober.go:28] interesting pod/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 20:28:19.063224 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:19.063134 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 20:28:20.062151 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:20.062119 2557 patch_prober.go:28] interesting pod/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 20:28:20.062561 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:20.062172 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 20:28:21.062037 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:21.062000 2557 patch_prober.go:28] interesting pod/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 20:28:21.062280 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:21.062062 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 20:28:22.061692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:22.061655 2557 patch_prober.go:28] interesting pod/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" start-of-body= Jul 15 20:28:22.061880 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:22.061709 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.52:15021/healthz/ready\": dial tcp 10.132.0.52:15021: connect: connection refused" Jul 15 20:28:23.065564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:23.065535 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:23.065936 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:23.065793 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:23.066624 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:23.066602 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:28:42.352983 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.352904 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7"] Jul 15 20:28:42.355633 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.355617 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:42.358507 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.358489 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:28:42.358606 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.358540 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 15 20:28:42.365455 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.365426 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7"] Jul 15 20:28:42.448480 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.448448 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4bfab-2444-4ff8-8236-c5bb0f29534f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7\" (UID: \"f4e4bfab-2444-4ff8-8236-c5bb0f29534f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:42.549380 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.549339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4bfab-2444-4ff8-8236-c5bb0f29534f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7\" (UID: \"f4e4bfab-2444-4ff8-8236-c5bb0f29534f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:42.549924 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.549904 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4bfab-2444-4ff8-8236-c5bb0f29534f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7\" (UID: \"f4e4bfab-2444-4ff8-8236-c5bb0f29534f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:42.667932 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.667844 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:42.817210 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:42.817177 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7"] Jul 15 20:28:42.819503 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:28:42.819478 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4e4bfab_2444_4ff8_8236_c5bb0f29534f.slice/crio-0f47814bbbed080fc1afb14e10cdb98302f80437d0a7c583cb4cfb806ee617e7 WatchSource:0}: Error finding container 0f47814bbbed080fc1afb14e10cdb98302f80437d0a7c583cb4cfb806ee617e7: Status 404 returned error can't find the container with id 0f47814bbbed080fc1afb14e10cdb98302f80437d0a7c583cb4cfb806ee617e7 Jul 15 20:28:43.817648 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:43.817557 2557 generic.go:356] "Generic (PLEG): container finished" podID="f4e4bfab-2444-4ff8-8236-c5bb0f29534f" containerID="0cb46f37e4c462114433dec63a21343af2898e6580ae8d4181688189a5950f12" exitCode=0 Jul 15 20:28:43.818016 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:43.817648 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" event={"ID":"f4e4bfab-2444-4ff8-8236-c5bb0f29534f","Type":"ContainerDied","Data":"0cb46f37e4c462114433dec63a21343af2898e6580ae8d4181688189a5950f12"} Jul 15 20:28:43.818016 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:43.817708 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" event={"ID":"f4e4bfab-2444-4ff8-8236-c5bb0f29534f","Type":"ContainerStarted","Data":"0f47814bbbed080fc1afb14e10cdb98302f80437d0a7c583cb4cfb806ee617e7"} Jul 15 20:28:44.965166 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:44.965141 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:44.974681 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:44.974658 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4bfab-2444-4ff8-8236-c5bb0f29534f-maas-api-service-ca\") pod \"f4e4bfab-2444-4ff8-8236-c5bb0f29534f\" (UID: \"f4e4bfab-2444-4ff8-8236-c5bb0f29534f\") " Jul 15 20:28:44.975047 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:44.975022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f4e4bfab-2444-4ff8-8236-c5bb0f29534f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "f4e4bfab-2444-4ff8-8236-c5bb0f29534f" (UID: "f4e4bfab-2444-4ff8-8236-c5bb0f29534f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:28:45.076432 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:45.076341 2557 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/f4e4bfab-2444-4ff8-8236-c5bb0f29534f-maas-api-service-ca\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:28:45.830362 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:45.830330 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" Jul 15 20:28:45.830629 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:45.830321 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7" event={"ID":"f4e4bfab-2444-4ff8-8236-c5bb0f29534f","Type":"ContainerDied","Data":"0f47814bbbed080fc1afb14e10cdb98302f80437d0a7c583cb4cfb806ee617e7"} Jul 15 20:28:45.830629 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:45.830468 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0f47814bbbed080fc1afb14e10cdb98302f80437d0a7c583cb4cfb806ee617e7" Jul 15 20:28:50.746759 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:50.746726 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:52.308638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:52.308602 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:53.161999 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:53.161964 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:53.264932 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:53.264899 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:53.481971 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:53.481929 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:57.462861 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:57.462830 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:28:59.396419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:28:59.396378 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:02.511593 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:02.511556 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:13.001078 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:13.001040 2557 scope.go:117] "RemoveContainer" containerID="43bab748f22b0161ea102d975ef3f30e9ebc5a5322bbff6979010376844bb0b3" Jul 15 20:29:20.061626 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:20.061591 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:20.125069 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:20.125031 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:39.113734 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:39.113696 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:40.414970 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:40.414939 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:46.245345 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:46.245303 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:46.313068 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:46.313025 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:49.732065 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:49.732028 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g"] Jul 15 20:29:49.732567 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:49.732330 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" containerID="cri-o://ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076" gracePeriod=30 Jul 15 20:29:49.757036 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:49.757007 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:49.982844 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:49.982755 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:51.012641 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:51.012605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:51.134053 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:51.134016 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:51.211529 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:51.211484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:51.992315 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:51.992279 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:52.086603 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:52.086570 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:52.195332 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:52.195300 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:53.068731 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:53.068697 2557 patch_prober.go:28] interesting pod/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:29:53.068923 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:53.068778 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:29:53.550672 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:53.550636 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:29:55.002228 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.002161 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:29:55.196882 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.196843 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e625929b-76dd-4fd7-91f7-00719062180f-istiod-ca-cert\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.196882 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.196890 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qxhw4\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-kube-api-access-qxhw4\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.196939 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-certs\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.196954 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-credential-socket\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.196987 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-envoy\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197003 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-socket\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197029 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-data\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197118 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e625929b-76dd-4fd7-91f7-00719062180f-istio-podinfo\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197448 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197172 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-istio-token\") pod \"e625929b-76dd-4fd7-91f7-00719062180f\" (UID: \"e625929b-76dd-4fd7-91f7-00719062180f\") " Jul 15 20:29:55.197448 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197295 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:29:55.197448 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197335 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e625929b-76dd-4fd7-91f7-00719062180f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:29:55.197597 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197438 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:29:55.197696 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197666 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.197824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197704 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.197824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.197719 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e625929b-76dd-4fd7-91f7-00719062180f-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.198208 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.198179 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:29:55.198351 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.198291 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-data" (OuterVolumeSpecName: "istio-data") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:29:55.199498 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.199469 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-kube-api-access-qxhw4" (OuterVolumeSpecName: "kube-api-access-qxhw4") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "kube-api-access-qxhw4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:29:55.199601 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.199493 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e625929b-76dd-4fd7-91f7-00719062180f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:29:55.199601 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.199519 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-istio-token" (OuterVolumeSpecName: "istio-token") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:29:55.199601 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.199518 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e625929b-76dd-4fd7-91f7-00719062180f" (UID: "e625929b-76dd-4fd7-91f7-00719062180f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:29:55.212759 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.212721 2557 generic.go:356] "Generic (PLEG): container finished" podID="e625929b-76dd-4fd7-91f7-00719062180f" containerID="ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076" exitCode=0 Jul 15 20:29:55.212867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.212763 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" event={"ID":"e625929b-76dd-4fd7-91f7-00719062180f","Type":"ContainerDied","Data":"ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076"} Jul 15 20:29:55.212867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.212815 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" Jul 15 20:29:55.212867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.212827 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g" event={"ID":"e625929b-76dd-4fd7-91f7-00719062180f","Type":"ContainerDied","Data":"30ed6ac0aa69167966bafa54117d856552ff8afc62d6b9fbc885959972d0f9a8"} Jul 15 20:29:55.212867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.212849 2557 scope.go:117] "RemoveContainer" containerID="ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076" Jul 15 20:29:55.222921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.222903 2557 scope.go:117] "RemoveContainer" containerID="94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b" Jul 15 20:29:55.232153 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.232136 2557 scope.go:117] "RemoveContainer" containerID="ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076" Jul 15 20:29:55.232425 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:29:55.232406 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076\": container with ID starting with ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076 not found: ID does not exist" containerID="ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076" Jul 15 20:29:55.232492 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.232433 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076"} err="failed to get container status \"ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076\": rpc error: code = NotFound desc = could not find container \"ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076\": container with ID starting with ae65e7c3044ba531f66f0679d2f6152edcd7a92558153408e91bcac009a25076 not found: ID does not exist" Jul 15 20:29:55.232492 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.232450 2557 scope.go:117] "RemoveContainer" containerID="94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b" Jul 15 20:29:55.232679 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:29:55.232665 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b\": container with ID starting with 94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b not found: ID does not exist" containerID="94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b" Jul 15 20:29:55.232722 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.232682 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b"} err="failed to get container status \"94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b\": rpc error: code = NotFound desc = could not find container \"94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b\": container with ID starting with 94dc48984f15bb4f566cc92042b2645f3eeddbb1b6bba54976c379a78d15655b not found: ID does not exist" Jul 15 20:29:55.242944 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.242919 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g"] Jul 15 20:29:55.246706 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.246684 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-83c1a2fd-openshift-default-58d47f5d6c-x7r8g"] Jul 15 20:29:55.298291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.298190 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e625929b-76dd-4fd7-91f7-00719062180f-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.298291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.298219 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.298291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.298227 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qxhw4\" (UniqueName: \"kubernetes.io/projected/e625929b-76dd-4fd7-91f7-00719062180f-kube-api-access-qxhw4\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.298291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.298264 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.298291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.298276 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:55.298291 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:55.298284 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e625929b-76dd-4fd7-91f7-00719062180f-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:29:56.785789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:29:56.785753 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e625929b-76dd-4fd7-91f7-00719062180f" path="/var/lib/kubelet/pods/e625929b-76dd-4fd7-91f7-00719062180f/volumes" Jul 15 20:30:00.082774 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.082734 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:00.156818 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.156787 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf"] Jul 15 20:30:00.157230 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157208 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" Jul 15 20:30:00.157230 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157222 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" Jul 15 20:30:00.157430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157261 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f4e4bfab-2444-4ff8-8236-c5bb0f29534f" containerName="revoke-keys" Jul 15 20:30:00.157430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157270 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="f4e4bfab-2444-4ff8-8236-c5bb0f29534f" containerName="revoke-keys" Jul 15 20:30:00.157430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157350 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" Jul 15 20:30:00.157430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157358 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" Jul 15 20:30:00.157430 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.157367 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="f4e4bfab-2444-4ff8-8236-c5bb0f29534f" containerName="revoke-keys" Jul 15 20:30:00.161824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.161808 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" Jul 15 20:30:00.164503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.164484 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-grvfv\"" Jul 15 20:30:00.168931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.168907 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf"] Jul 15 20:30:00.171995 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.171968 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:00.173131 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.173110 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" Jul 15 20:30:00.321615 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.321584 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf"] Jul 15 20:30:00.323038 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:30:00.323014 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8707a2ed_31bf_443b_a585_8b760e22f1e4.slice/crio-ca7dec786114adef8b47fc69db8f195317e668bce3c88d4b076874ea3747e876 WatchSource:0}: Error finding container ca7dec786114adef8b47fc69db8f195317e668bce3c88d4b076874ea3747e876: Status 404 returned error can't find the container with id ca7dec786114adef8b47fc69db8f195317e668bce3c88d4b076874ea3747e876 Jul 15 20:30:00.789129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:00.789100 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:01.252995 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:01.252959 2557 generic.go:356] "Generic (PLEG): container finished" podID="8707a2ed-31bf-443b-a585-8b760e22f1e4" containerID="002dc00fb66db79e63382857a3037f90ca96a99760b37c5ba73af5c083838df5" exitCode=0 Jul 15 20:30:01.253417 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:01.253043 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" event={"ID":"8707a2ed-31bf-443b-a585-8b760e22f1e4","Type":"ContainerDied","Data":"002dc00fb66db79e63382857a3037f90ca96a99760b37c5ba73af5c083838df5"} Jul 15 20:30:01.253417 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:01.253092 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" event={"ID":"8707a2ed-31bf-443b-a585-8b760e22f1e4","Type":"ContainerStarted","Data":"ca7dec786114adef8b47fc69db8f195317e668bce3c88d4b076874ea3747e876"} Jul 15 20:30:02.401812 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:02.401786 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" Jul 15 20:30:03.265366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:03.265327 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" event={"ID":"8707a2ed-31bf-443b-a585-8b760e22f1e4","Type":"ContainerDied","Data":"ca7dec786114adef8b47fc69db8f195317e668bce3c88d4b076874ea3747e876"} Jul 15 20:30:03.265366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:03.265344 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29735790-6lslf" Jul 15 20:30:03.265586 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:03.265381 2557 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ca7dec786114adef8b47fc69db8f195317e668bce3c88d4b076874ea3747e876" Jul 15 20:30:07.082837 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:07.082804 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:07.186345 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:07.186297 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:07.291773 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:07.291746 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:17.798756 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:17.798703 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:18.143140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:18.143035 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:39.129628 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:39.129595 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:41.557363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:41.557328 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:41.736617 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:41.736586 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:45.284230 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:45.284192 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:45.383483 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:45.383449 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:45.633960 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:45.633867 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:46.013131 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.013092 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw"] Jul 15 20:30:46.013812 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.013791 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8707a2ed-31bf-443b-a585-8b760e22f1e4" containerName="cleanup" Jul 15 20:30:46.013868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.013818 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="8707a2ed-31bf-443b-a585-8b760e22f1e4" containerName="cleanup" Jul 15 20:30:46.013868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.013847 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" Jul 15 20:30:46.013868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.013856 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="e625929b-76dd-4fd7-91f7-00719062180f" containerName="istio-proxy" Jul 15 20:30:46.013986 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.013974 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="8707a2ed-31bf-443b-a585-8b760e22f1e4" containerName="cleanup" Jul 15 20:30:46.015863 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.015835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:46.017760 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.017744 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.024886 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.024800 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-bec571-openshift-default-dockercfg-9przh\"" Jul 15 20:30:46.045011 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.044978 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw"] Jul 15 20:30:46.106736 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106697 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-credential-socket\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.106967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-envoy\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.106967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106812 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-certs\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.106967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106837 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-data\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.106967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106878 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-podinfo\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.106967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106928 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a0bbd8a-6697-491c-89d2-da24faee0626-istiod-ca-cert\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.106967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.106950 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-socket\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.107187 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.107007 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79gd2\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-kube-api-access-79gd2\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.107187 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.107049 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-token\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.158343 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.158302 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:46.207824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.207744 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a0bbd8a-6697-491c-89d2-da24faee0626-istiod-ca-cert\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.207824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.207782 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-socket\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.207824 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.207803 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79gd2\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-kube-api-access-79gd2\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208148 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.207951 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-token\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208223 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208197 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-credential-socket\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208223 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208211 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-socket\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208370 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208262 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-envoy\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208370 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208339 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-certs\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208370 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208361 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-data\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208522 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208491 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-credential-socket\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208573 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208540 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-podinfo\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208679 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208654 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a0bbd8a-6697-491c-89d2-da24faee0626-istiod-ca-cert\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208727 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208664 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-certs\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.208727 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.208716 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-data\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.211305 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.211260 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-envoy\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.211905 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.211877 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-podinfo\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.216913 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.216889 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-token\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.217479 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.217462 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79gd2\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-kube-api-access-79gd2\") pod \"e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.328560 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.328480 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:46.480229 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.480200 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw"] Jul 15 20:30:46.483083 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:30:46.483052 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a0bbd8a_6697_491c_89d2_da24faee0626.slice/crio-e66d53dbad23e7bfa9341f6980bec3d291a07819c84df34ae281716cfbfc40a4 WatchSource:0}: Error finding container e66d53dbad23e7bfa9341f6980bec3d291a07819c84df34ae281716cfbfc40a4: Status 404 returned error can't find the container with id e66d53dbad23e7bfa9341f6980bec3d291a07819c84df34ae281716cfbfc40a4 Jul 15 20:30:46.485496 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.485450 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:30:46.485580 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.485550 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:30:46.485619 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.485598 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:30:46.500882 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:46.500855 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" event={"ID":"6a0bbd8a-6697-491c-89d2-da24faee0626","Type":"ContainerStarted","Data":"e66d53dbad23e7bfa9341f6980bec3d291a07819c84df34ae281716cfbfc40a4"} Jul 15 20:30:47.509765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:47.509721 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" event={"ID":"6a0bbd8a-6697-491c-89d2-da24faee0626","Type":"ContainerStarted","Data":"e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8"} Jul 15 20:30:47.536196 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:47.536147 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podStartSLOduration=2.536131326 podStartE2EDuration="2.536131326s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:30:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:30:47.535021785 +0000 UTC m=+2195.205808323" watchObservedRunningTime="2026-07-15 20:30:47.536131326 +0000 UTC m=+2195.206917847" Jul 15 20:30:48.329566 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:48.329523 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:48.334576 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:48.334543 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:48.515888 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:48.515852 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:48.517016 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:48.516998 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:30:49.588005 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:49.587956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:49.705677 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:49.705634 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:49.792305 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:49.792263 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:50.225150 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:50.225113 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:50.421679 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:50.421633 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:50.529309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:50.529185 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:54.013662 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.013602 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:54.324712 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.324605 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:54.735899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.735863 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs"] Jul 15 20:30:54.739866 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.739845 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.743863 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.743841 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-4d97ce-openshift-default-dockercfg-2k6ft\"" Jul 15 20:30:54.763509 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.763484 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:54.774621 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.774600 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs"] Jul 15 20:30:54.800644 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800606 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-envoy\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800644 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800638 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-data\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800661 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-token\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800715 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-podinfo\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800758 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istiod-ca-cert\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800782 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-certs\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800840 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvnnc\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-kube-api-access-kvnnc\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.800867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800869 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-socket\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.801141 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.800910 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-credential-socket\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902006 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.901968 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-credential-socket\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902194 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902070 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-envoy\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902194 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902094 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-data\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902194 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902114 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-token\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902194 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-podinfo\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902194 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902155 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istiod-ca-cert\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902504 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902312 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-certs\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902504 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902385 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvnnc\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-kube-api-access-kvnnc\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902504 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902429 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-socket\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902504 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902446 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-credential-socket\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902722 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902568 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-data\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902778 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902745 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-socket\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902778 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902767 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-certs\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.902889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.902828 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istiod-ca-cert\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.904561 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.904541 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-envoy\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.904769 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.904738 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-podinfo\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.912108 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.912072 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvnnc\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-kube-api-access-kvnnc\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:54.912885 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:54.912866 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-token\") pod \"e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:55.051188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.051097 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:55.088634 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.088603 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:55.208459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.208432 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs"] Jul 15 20:30:55.210040 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:30:55.210002 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c40f77c_f4cf_45f9_b107_f6ed716e5441.slice/crio-2a4db1146d02dd93292f653c904c170791c17d0c5138ba5df063b25b7d8efaac WatchSource:0}: Error finding container 2a4db1146d02dd93292f653c904c170791c17d0c5138ba5df063b25b7d8efaac: Status 404 returned error can't find the container with id 2a4db1146d02dd93292f653c904c170791c17d0c5138ba5df063b25b7d8efaac Jul 15 20:30:55.212352 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.212318 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:30:55.212459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.212386 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:30:55.212459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.212415 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:30:55.559007 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.558969 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" event={"ID":"2c40f77c-f4cf-45f9-b107-f6ed716e5441","Type":"ContainerStarted","Data":"7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85"} Jul 15 20:30:55.559186 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.559029 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" event={"ID":"2c40f77c-f4cf-45f9-b107-f6ed716e5441","Type":"ContainerStarted","Data":"2a4db1146d02dd93292f653c904c170791c17d0c5138ba5df063b25b7d8efaac"} Jul 15 20:30:55.586081 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:55.586029 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podStartSLOduration=1.586012269 podStartE2EDuration="1.586012269s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:30:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:30:55.58250641 +0000 UTC m=+2203.253292934" watchObservedRunningTime="2026-07-15 20:30:55.586012269 +0000 UTC m=+2203.256798794" Jul 15 20:30:56.052136 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:56.052096 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:56.057531 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:56.057506 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:56.565305 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:56.565268 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:56.566195 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:56.566175 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:30:57.567114 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:57.567073 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:57.661577 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:57.661542 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:57.725384 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:57.725350 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:57.829597 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:57.829498 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:58.302584 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:58.302545 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:30:58.421829 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:30:58.421781 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:02.100392 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:02.100347 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:04.559459 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:04.559408 2557 patch_prober.go:28] interesting pod/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 20:31:04.559973 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:04.559508 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 20:31:04.636283 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:04.635824 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:06.650323 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:06.650285 2557 generic.go:356] "Generic (PLEG): container finished" podID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerID="e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8" exitCode=0 Jul 15 20:31:06.650762 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:06.650355 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" event={"ID":"6a0bbd8a-6697-491c-89d2-da24faee0626","Type":"ContainerDied","Data":"e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8"} Jul 15 20:31:06.650810 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:06.650792 2557 scope.go:117] "RemoveContainer" containerID="e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8" Jul 15 20:31:06.651985 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:06.651954 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:31:06.652075 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:06.652034 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:31:06.652075 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:06.652064 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:31:07.662655 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:07.662602 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" event={"ID":"6a0bbd8a-6697-491c-89d2-da24faee0626","Type":"ContainerStarted","Data":"f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0"} Jul 15 20:31:08.329319 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:08.329279 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:31:08.329702 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:08.329679 2557 patch_prober.go:28] interesting pod/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 20:31:08.329885 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:08.329847 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 20:31:09.329308 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:09.329263 2557 patch_prober.go:28] interesting pod/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 20:31:09.329833 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:09.329345 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 20:31:10.329358 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:10.329314 2557 patch_prober.go:28] interesting pod/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 20:31:10.329954 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:10.329383 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 20:31:11.332265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:11.330410 2557 patch_prober.go:28] interesting pod/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" start-of-body= Jul 15 20:31:11.332265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:11.330496 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.55:15021/healthz/ready\": dial tcp 10.132.0.55:15021: connect: connection refused" Jul 15 20:31:11.367665 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:11.367632 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:11.454397 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:11.454362 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:12.333525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.333495 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:31:12.334949 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.333921 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:31:12.334949 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.334536 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:31:12.378698 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.378667 2557 patch_prober.go:28] interesting pod/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:48788->10.132.0.56:15021: read: connection reset by peer" start-of-body= Jul 15 20:31:12.378827 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.378752 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": read tcp 10.132.0.2:48788->10.132.0.56:15021: read: connection reset by peer" Jul 15 20:31:12.694020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.693984 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerID="7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85" exitCode=0 Jul 15 20:31:12.694249 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.694060 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" event={"ID":"2c40f77c-f4cf-45f9-b107-f6ed716e5441","Type":"ContainerDied","Data":"7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85"} Jul 15 20:31:12.694609 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.694592 2557 scope.go:117] "RemoveContainer" containerID="7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85" Jul 15 20:31:12.695643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.695615 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:31:12.695734 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.695684 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:31:12.695734 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:12.695717 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:31:13.034958 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:13.034862 2557 scope.go:117] "RemoveContainer" containerID="4ba52b0a9fddb98c04645a854cabd3d011cbc0e9adc8efa50ba17bb542395faa" Jul 15 20:31:13.702060 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:13.702016 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" event={"ID":"2c40f77c-f4cf-45f9-b107-f6ed716e5441","Type":"ContainerStarted","Data":"3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20"} Jul 15 20:31:14.051851 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:14.051754 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:31:14.052028 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:14.052007 2557 patch_prober.go:28] interesting pod/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 20:31:14.052074 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:14.052055 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 20:31:15.052434 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:15.052393 2557 patch_prober.go:28] interesting pod/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 20:31:15.053022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:15.052992 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 20:31:16.051770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:16.051728 2557 patch_prober.go:28] interesting pod/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 20:31:16.051967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:16.051791 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 20:31:17.052057 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:17.052016 2557 patch_prober.go:28] interesting pod/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" start-of-body= Jul 15 20:31:17.052551 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:17.052078 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.56:15021/healthz/ready\": dial tcp 10.132.0.56:15021: connect: connection refused" Jul 15 20:31:18.055575 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:18.055546 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:31:18.056066 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:18.055950 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:31:18.056660 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:18.056641 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:31:19.425516 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:19.425483 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:19.529783 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:19.529746 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:32.252460 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:32.252429 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:31:55.619203 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:31:55.619171 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:06.188270 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:06.188219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:07.292260 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:07.292203 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:07.898917 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:07.898877 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:07.998817 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:07.998785 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:17.960120 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:17.960084 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:22.456851 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:22.456809 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:29.021573 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:29.021530 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:29.122350 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:29.122311 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:33.167163 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:33.167125 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw"] Jul 15 20:32:33.167612 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:33.167353 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" containerID="cri-o://f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0" gracePeriod=30 Jul 15 20:32:33.582056 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:33.582010 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:33.678469 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:33.678426 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:34.216609 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:34.216574 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:34.312359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:34.312284 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:38.432219 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.432143 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:32:38.577957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.577915 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-certs\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.577974 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-data\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578055 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-socket\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578081 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-79gd2\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-kube-api-access-79gd2\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578101 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a0bbd8a-6697-491c-89d2-da24faee0626-istiod-ca-cert\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578118 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-envoy\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578452 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578170 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-credential-socket\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578452 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578191 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-token\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578452 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578218 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-podinfo\") pod \"6a0bbd8a-6697-491c-89d2-da24faee0626\" (UID: \"6a0bbd8a-6697-491c-89d2-da24faee0626\") " Jul 15 20:32:38.578452 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578269 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:32:38.578674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578451 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:32:38.578674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578562 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a0bbd8a-6697-491c-89d2-da24faee0626-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:32:38.578674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578631 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.578674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578653 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.578919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.578862 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:32:38.579133 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.579022 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-data" (OuterVolumeSpecName: "istio-data") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:32:38.580711 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.580680 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:32:38.580711 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.580699 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:32:38.580711 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.580686 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-token" (OuterVolumeSpecName: "istio-token") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:32:38.580901 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.580771 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-kube-api-access-79gd2" (OuterVolumeSpecName: "kube-api-access-79gd2") pod "6a0bbd8a-6697-491c-89d2-da24faee0626" (UID: "6a0bbd8a-6697-491c-89d2-da24faee0626"). InnerVolumeSpecName "kube-api-access-79gd2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:32:38.679957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679920 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.679957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679949 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-79gd2\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-kube-api-access-79gd2\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.679957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679965 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6a0bbd8a-6697-491c-89d2-da24faee0626-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.680209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679975 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.680209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679983 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6a0bbd8a-6697-491c-89d2-da24faee0626-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.680209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679991 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:38.680209 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:38.679999 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6a0bbd8a-6697-491c-89d2-da24faee0626-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:32:39.195638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.195606 2557 generic.go:356] "Generic (PLEG): container finished" podID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerID="f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0" exitCode=0 Jul 15 20:32:39.195821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.195658 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" Jul 15 20:32:39.195821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.195691 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" event={"ID":"6a0bbd8a-6697-491c-89d2-da24faee0626","Type":"ContainerDied","Data":"f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0"} Jul 15 20:32:39.195821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.195748 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw" event={"ID":"6a0bbd8a-6697-491c-89d2-da24faee0626","Type":"ContainerDied","Data":"e66d53dbad23e7bfa9341f6980bec3d291a07819c84df34ae281716cfbfc40a4"} Jul 15 20:32:39.195821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.195779 2557 scope.go:117] "RemoveContainer" containerID="f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0" Jul 15 20:32:39.205656 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.205638 2557 scope.go:117] "RemoveContainer" containerID="e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8" Jul 15 20:32:39.214875 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.214844 2557 scope.go:117] "RemoveContainer" containerID="f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0" Jul 15 20:32:39.215203 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:32:39.215177 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0\": container with ID starting with f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0 not found: ID does not exist" containerID="f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0" Jul 15 20:32:39.215318 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.215212 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0"} err="failed to get container status \"f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0\": rpc error: code = NotFound desc = could not find container \"f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0\": container with ID starting with f3dc82b94a9c74c0184de889790e3435da74544324cff77e7e744755c52bacd0 not found: ID does not exist" Jul 15 20:32:39.215318 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.215231 2557 scope.go:117] "RemoveContainer" containerID="e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8" Jul 15 20:32:39.215545 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:32:39.215526 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8\": container with ID starting with e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8 not found: ID does not exist" containerID="e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8" Jul 15 20:32:39.215610 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.215555 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8"} err="failed to get container status \"e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8\": rpc error: code = NotFound desc = could not find container \"e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8\": container with ID starting with e532522f68bf64a0bd8b92557c2ac1bf9d7a574f39e38452843026a793616aa8 not found: ID does not exist" Jul 15 20:32:39.216686 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.216668 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw"] Jul 15 20:32:39.220755 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:39.220723 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-bec571-openshift-default-8659d5ff99-bjrkw"] Jul 15 20:32:40.785744 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:40.785706 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" path="/var/lib/kubelet/pods/6a0bbd8a-6697-491c-89d2-da24faee0626/volumes" Jul 15 20:32:44.075781 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:44.075737 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:48.646427 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:48.646395 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:55.133932 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:55.133895 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:55.234010 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:55.233976 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:59.382492 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:59.382456 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs"] Jul 15 20:32:59.383009 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:59.382669 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" containerID="cri-o://3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20" gracePeriod=30 Jul 15 20:32:59.420219 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:59.420183 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:32:59.691514 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:32:59.691475 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:00.086197 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.086087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:00.503487 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.503456 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75"] Jul 15 20:33:00.503997 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.503979 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" Jul 15 20:33:00.504055 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.504000 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" Jul 15 20:33:00.504055 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.504021 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" Jul 15 20:33:00.504055 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.504027 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" Jul 15 20:33:00.504154 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.504112 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" Jul 15 20:33:00.507542 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.507523 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.511977 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.511954 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-514f9c-openshift-default-dockercfg-cqp9m\"" Jul 15 20:33:00.526808 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.526778 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75"] Jul 15 20:33:00.540342 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.540309 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:00.609317 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-certs\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609512 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609353 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-envoy\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609512 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609407 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-token\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609512 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609433 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6v6c\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-kube-api-access-l6v6c\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609694 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609588 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-data\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609694 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609630 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-credential-socket\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609796 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609726 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-podinfo\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609853 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609789 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f58bdb6-1037-42a7-88bd-8251e697b24e-istiod-ca-cert\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.609853 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.609828 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-socket\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.710899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.710865 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-podinfo\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.710936 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f58bdb6-1037-42a7-88bd-8251e697b24e-istiod-ca-cert\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.710973 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-socket\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711012 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-certs\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711054 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-envoy\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711095 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-token\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711119 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6v6c\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-kube-api-access-l6v6c\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-data\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711295 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-credential-socket\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711584 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711476 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-socket\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711584 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711545 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-certs\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711696 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711664 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-credential-socket\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711857 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711813 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f58bdb6-1037-42a7-88bd-8251e697b24e-istiod-ca-cert\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.711970 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.711933 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-data\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.713658 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.713635 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-podinfo\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.713900 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.713883 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-envoy\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.721167 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.721141 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-token\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.722015 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.721993 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6v6c\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-kube-api-access-l6v6c\") pod \"e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.773766 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.773680 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:00.819434 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.819397 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:00.974419 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.974389 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75"] Jul 15 20:33:00.975630 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:33:00.975602 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f58bdb6_1037_42a7_88bd_8251e697b24e.slice/crio-9f6733256a2b0a1df6f7dd500181227ec03c6d8fef31d0aef43698f922d0ade9 WatchSource:0}: Error finding container 9f6733256a2b0a1df6f7dd500181227ec03c6d8fef31d0aef43698f922d0ade9: Status 404 returned error can't find the container with id 9f6733256a2b0a1df6f7dd500181227ec03c6d8fef31d0aef43698f922d0ade9 Jul 15 20:33:00.978194 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.978162 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:00.978296 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.978230 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:00.978350 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:00.978324 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:01.327096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:01.327004 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" event={"ID":"5f58bdb6-1037-42a7-88bd-8251e697b24e","Type":"ContainerStarted","Data":"25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6"} Jul 15 20:33:01.327096 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:01.327066 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" event={"ID":"5f58bdb6-1037-42a7-88bd-8251e697b24e","Type":"ContainerStarted","Data":"9f6733256a2b0a1df6f7dd500181227ec03c6d8fef31d0aef43698f922d0ade9"} Jul 15 20:33:01.356551 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:01.356490 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podStartSLOduration=1.356474889 podStartE2EDuration="1.356474889s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:33:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:33:01.352852397 +0000 UTC m=+2329.023638928" watchObservedRunningTime="2026-07-15 20:33:01.356474889 +0000 UTC m=+2329.027261414" Jul 15 20:33:01.819827 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:01.819790 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:02.825977 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:02.825946 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:03.058164 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.058124 2557 patch_prober.go:28] interesting pod/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:33:03.058363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.058185 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:33:03.338959 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.338931 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:03.339889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.339863 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:03.388532 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.388432 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:03.482048 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.482015 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:03.627493 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.627393 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:03.829962 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:03.829927 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:04.193481 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.193446 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:04.297298 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.297269 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:04.646363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.646336 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:33:04.757759 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.757673 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-podinfo\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.757759 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.757737 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-token\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.757954 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.757815 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-credential-socket\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.757954 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.757850 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-certs\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.757954 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.757880 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istiod-ca-cert\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.758083 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.757967 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-data\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.758083 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758001 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kvnnc\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-kube-api-access-kvnnc\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.758083 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758034 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-socket\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.758083 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758059 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-envoy\") pod \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\" (UID: \"2c40f77c-f4cf-45f9-b107-f6ed716e5441\") " Jul 15 20:33:04.758321 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758133 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:04.758321 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758151 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:04.758510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758379 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:04.758581 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758556 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.758581 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758570 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.758581 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758579 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.758731 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.758617 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:33:04.759381 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.759348 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:04.760349 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.760319 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:33:04.760349 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.760345 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:33:04.760502 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.760442 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:04.760651 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.760627 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-kube-api-access-kvnnc" (OuterVolumeSpecName: "kube-api-access-kvnnc") pod "2c40f77c-f4cf-45f9-b107-f6ed716e5441" (UID: "2c40f77c-f4cf-45f9-b107-f6ed716e5441"). InnerVolumeSpecName "kube-api-access-kvnnc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:33:04.859551 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.859518 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.859551 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.859547 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.859551 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.859556 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kvnnc\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-kube-api-access-kvnnc\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.859972 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.859565 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.859972 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.859577 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:04.859972 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:04.859586 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c40f77c-f4cf-45f9-b107-f6ed716e5441-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:05.351103 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.351060 2557 generic.go:356] "Generic (PLEG): container finished" podID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerID="3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20" exitCode=0 Jul 15 20:33:05.351348 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.351115 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" Jul 15 20:33:05.351348 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.351151 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" event={"ID":"2c40f77c-f4cf-45f9-b107-f6ed716e5441","Type":"ContainerDied","Data":"3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20"} Jul 15 20:33:05.351348 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.351230 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs" event={"ID":"2c40f77c-f4cf-45f9-b107-f6ed716e5441","Type":"ContainerDied","Data":"2a4db1146d02dd93292f653c904c170791c17d0c5138ba5df063b25b7d8efaac"} Jul 15 20:33:05.351348 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.351286 2557 scope.go:117] "RemoveContainer" containerID="3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20" Jul 15 20:33:05.368959 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.368933 2557 scope.go:117] "RemoveContainer" containerID="7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85" Jul 15 20:33:05.378386 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.378365 2557 scope.go:117] "RemoveContainer" containerID="3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20" Jul 15 20:33:05.381777 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:33:05.378909 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20\": container with ID starting with 3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20 not found: ID does not exist" containerID="3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20" Jul 15 20:33:05.381777 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.378949 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20"} err="failed to get container status \"3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20\": rpc error: code = NotFound desc = could not find container \"3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20\": container with ID starting with 3ab54c78a514e15a9e6280b7b76c859a31cfc972ac0e639416f539e52cfc3f20 not found: ID does not exist" Jul 15 20:33:05.381777 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.378990 2557 scope.go:117] "RemoveContainer" containerID="7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85" Jul 15 20:33:05.381777 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:33:05.379456 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85\": container with ID starting with 7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85 not found: ID does not exist" containerID="7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85" Jul 15 20:33:05.381777 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.379481 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85"} err="failed to get container status \"7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85\": rpc error: code = NotFound desc = could not find container \"7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85\": container with ID starting with 7ad0297a62527755e48c8f071969e87b43d60b4c66df2acd45461a4e26998a85 not found: ID does not exist" Jul 15 20:33:05.386900 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.386877 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs"] Jul 15 20:33:05.394730 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:05.394706 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-4d97ce-openshift-default-7d5dfd6f7f-zqlxs"] Jul 15 20:33:06.787251 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:06.787209 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" path="/var/lib/kubelet/pods/2c40f77c-f4cf-45f9-b107-f6ed716e5441/volumes" Jul 15 20:33:07.942665 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:07.942612 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:08.470143 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.470111 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:08.922794 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.922713 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj"] Jul 15 20:33:08.923186 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.923174 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" Jul 15 20:33:08.923230 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.923188 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" Jul 15 20:33:08.923230 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.923197 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" Jul 15 20:33:08.923230 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.923203 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" Jul 15 20:33:08.923349 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.923292 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" Jul 15 20:33:08.923349 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.923302 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a0bbd8a-6697-491c-89d2-da24faee0626" containerName="istio-proxy" Jul 15 20:33:08.928008 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.927991 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:08.935173 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.935149 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-76c4bb-openshift-default-dockercfg-qnxmr\"" Jul 15 20:33:08.936763 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.936593 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:08.959921 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:08.959894 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj"] Jul 15 20:33:09.004700 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004665 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4d4d\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-kube-api-access-n4d4d\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.004877 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004737 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-envoy\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.004877 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004784 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-certs\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.004877 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004811 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-socket\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.004877 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004871 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9665df5c-7a84-4f82-99f1-44367313a229-istio-podinfo\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.005022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004934 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-credential-socket\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.005022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004960 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-data\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.005022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.004988 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9665df5c-7a84-4f82-99f1-44367313a229-istiod-ca-cert\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.005112 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.005042 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-istio-token\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106425 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106373 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4d4d\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-kube-api-access-n4d4d\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106607 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106463 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-envoy\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106607 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106491 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-certs\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106607 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106516 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-socket\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106607 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106561 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9665df5c-7a84-4f82-99f1-44367313a229-istio-podinfo\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106607 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106598 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-credential-socket\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106888 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106632 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-data\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106888 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106661 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9665df5c-7a84-4f82-99f1-44367313a229-istiod-ca-cert\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.106888 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106703 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-istio-token\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.107039 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.106989 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-certs\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.107088 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.107049 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-data\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.107136 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.107111 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-credential-socket\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.107294 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.107270 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-socket\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.107426 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.107410 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9665df5c-7a84-4f82-99f1-44367313a229-istiod-ca-cert\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.109521 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.109493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-envoy\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.109747 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.109726 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9665df5c-7a84-4f82-99f1-44367313a229-istio-podinfo\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.115909 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.115884 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4d4d\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-kube-api-access-n4d4d\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.116534 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.116517 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-istio-token\") pod \"e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.240663 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.240612 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:09.418352 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.418317 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj"] Jul 15 20:33:09.430704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.429622 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:09.430704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.429872 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:09.430704 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.429950 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:09.466654 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:09.466628 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:10.006802 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:10.006768 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:10.382603 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:10.382501 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" event={"ID":"9665df5c-7a84-4f82-99f1-44367313a229","Type":"ContainerStarted","Data":"c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718"} Jul 15 20:33:10.382603 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:10.382593 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" event={"ID":"9665df5c-7a84-4f82-99f1-44367313a229","Type":"ContainerStarted","Data":"2772116bf8e5f0bfb71a9460c272ec4556186eebe59f156dc4c1674b5a82f894"} Jul 15 20:33:10.411478 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:10.411406 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podStartSLOduration=2.411385535 podStartE2EDuration="2.411385535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:33:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:33:10.409318227 +0000 UTC m=+2338.080104753" watchObservedRunningTime="2026-07-15 20:33:10.411385535 +0000 UTC m=+2338.082172052" Jul 15 20:33:11.241144 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:11.241100 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:11.246382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:11.246351 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:11.389411 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:11.389375 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:11.390320 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:11.390297 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:12.492296 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:12.492258 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:12.655899 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:12.655862 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:12.757083 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:12.756989 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:13.206023 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:13.205984 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:13.309550 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:13.309509 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:13.409915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:13.409808 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:17.125617 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.125573 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:17.425372 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.425345 2557 generic.go:356] "Generic (PLEG): container finished" podID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerID="25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6" exitCode=0 Jul 15 20:33:17.425536 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.425416 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" event={"ID":"5f58bdb6-1037-42a7-88bd-8251e697b24e","Type":"ContainerDied","Data":"25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6"} Jul 15 20:33:17.425883 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.425870 2557 scope.go:117] "RemoveContainer" containerID="25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6" Jul 15 20:33:17.427020 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.426990 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:17.427092 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.427076 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:17.427136 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:17.427119 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:18.339484 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:18.339441 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:18.444528 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:18.444493 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" event={"ID":"5f58bdb6-1037-42a7-88bd-8251e697b24e","Type":"ContainerStarted","Data":"f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d"} Jul 15 20:33:18.820641 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:18.820598 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:18.822630 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:18.822592 2557 patch_prober.go:28] interesting pod/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 20:33:18.822795 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:18.822654 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 20:33:19.820828 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:19.820787 2557 patch_prober.go:28] interesting pod/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 20:33:19.821453 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:19.821419 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 20:33:20.820266 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:20.820201 2557 patch_prober.go:28] interesting pod/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 20:33:20.820500 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:20.820303 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 20:33:21.819911 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:21.819858 2557 patch_prober.go:28] interesting pod/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" start-of-body= Jul 15 20:33:21.820469 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:21.819934 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.57:15021/healthz/ready\": dial tcp 10.132.0.57:15021: connect: connection refused" Jul 15 20:33:22.829684 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:22.829643 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:22.831322 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:22.831295 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:22.834101 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:22.834072 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:25.197629 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:25.197586 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:25.305716 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:25.305683 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:27.430488 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:27.430441 2557 patch_prober.go:28] interesting pod/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 15 20:33:27.430932 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:27.430520 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 15 20:33:28.503376 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:28.503344 2557 generic.go:356] "Generic (PLEG): container finished" podID="9665df5c-7a84-4f82-99f1-44367313a229" containerID="c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718" exitCode=0 Jul 15 20:33:28.503789 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:28.503414 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" event={"ID":"9665df5c-7a84-4f82-99f1-44367313a229","Type":"ContainerDied","Data":"c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718"} Jul 15 20:33:28.503837 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:28.503821 2557 scope.go:117] "RemoveContainer" containerID="c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718" Jul 15 20:33:28.504938 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:28.504908 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:28.505018 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:28.504979 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:28.505018 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:28.505011 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:33:29.513727 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:29.513680 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" event={"ID":"9665df5c-7a84-4f82-99f1-44367313a229","Type":"ContainerStarted","Data":"92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9"} Jul 15 20:33:30.241561 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:30.241523 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:30.241841 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:30.241816 2557 patch_prober.go:28] interesting pod/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 20:33:30.241937 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:30.241873 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 20:33:31.241894 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:31.241849 2557 patch_prober.go:28] interesting pod/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 20:33:31.242325 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:31.241931 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 20:33:32.241195 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:32.241156 2557 patch_prober.go:28] interesting pod/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 20:33:32.241392 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:32.241217 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 20:33:33.241936 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:33.241888 2557 patch_prober.go:28] interesting pod/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" start-of-body= Jul 15 20:33:33.242374 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:33.241974 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.58:15021/healthz/ready\": dial tcp 10.132.0.58:15021: connect: connection refused" Jul 15 20:33:34.245656 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:34.245624 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:34.246143 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:34.245907 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:34.246821 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:34.246803 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:33:34.349126 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:34.349087 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:34.457638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:34.457606 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:36.411825 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:36.411793 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:36.511021 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:36.510967 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:36.761640 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:36.761607 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:40.734869 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:40.734834 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:47.347918 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:47.347880 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:47.436758 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:47.436717 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:49.148599 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:49.148564 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:49.250816 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:49.250782 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:51.637517 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:51.637476 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:51.656875 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:51.656835 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75"] Jul 15 20:33:51.657131 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:51.657094 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" containerID="cri-o://f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d" gracePeriod=30 Jul 15 20:33:51.750960 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:51.750926 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:52.833762 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:52.833723 2557 patch_prober.go:28] interesting pod/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:33:52.834154 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:52.833785 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:33:53.152987 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:53.152835 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:55.552919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:55.552885 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:33:56.923651 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.923584 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:56.935345 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935312 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f58bdb6-1037-42a7-88bd-8251e697b24e-istiod-ca-cert\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935369 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-podinfo\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935404 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-socket\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935445 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-envoy\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935470 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-credential-socket\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935525 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935525 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-data\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935804 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935552 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-certs\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935804 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935596 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l6v6c\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-kube-api-access-l6v6c\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935804 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935630 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-token\") pod \"5f58bdb6-1037-42a7-88bd-8251e697b24e\" (UID: \"5f58bdb6-1037-42a7-88bd-8251e697b24e\") " Jul 15 20:33:56.935804 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935730 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f58bdb6-1037-42a7-88bd-8251e697b24e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:33:56.935804 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935729 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:56.936059 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935838 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:56.936059 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.935908 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:56.936290 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.936261 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:56.936589 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.936293 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:56.936589 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.936308 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:56.936589 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.936322 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f58bdb6-1037-42a7-88bd-8251e697b24e-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:56.936831 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.936801 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:56.938048 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.938015 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:33:56.938149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.938043 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:33:56.938149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.938051 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:33:56.938149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:56.938088 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-kube-api-access-l6v6c" (OuterVolumeSpecName: "kube-api-access-l6v6c") pod "5f58bdb6-1037-42a7-88bd-8251e697b24e" (UID: "5f58bdb6-1037-42a7-88bd-8251e697b24e"). InnerVolumeSpecName "kube-api-access-l6v6c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:33:57.036897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.036862 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:57.036897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.036891 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:57.036897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.036900 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:57.036897 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.036909 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l6v6c\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-kube-api-access-l6v6c\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:57.037180 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.036917 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f58bdb6-1037-42a7-88bd-8251e697b24e-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:33:57.689366 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.689331 2557 generic.go:356] "Generic (PLEG): container finished" podID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerID="f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d" exitCode=0 Jul 15 20:33:57.689564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.689378 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" Jul 15 20:33:57.689564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.689398 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" event={"ID":"5f58bdb6-1037-42a7-88bd-8251e697b24e","Type":"ContainerDied","Data":"f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d"} Jul 15 20:33:57.689564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.689443 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75" event={"ID":"5f58bdb6-1037-42a7-88bd-8251e697b24e","Type":"ContainerDied","Data":"9f6733256a2b0a1df6f7dd500181227ec03c6d8fef31d0aef43698f922d0ade9"} Jul 15 20:33:57.689564 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.689466 2557 scope.go:117] "RemoveContainer" containerID="f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d" Jul 15 20:33:57.699904 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.699881 2557 scope.go:117] "RemoveContainer" containerID="25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6" Jul 15 20:33:57.708479 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.708458 2557 scope.go:117] "RemoveContainer" containerID="f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d" Jul 15 20:33:57.708779 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:33:57.708759 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d\": container with ID starting with f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d not found: ID does not exist" containerID="f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d" Jul 15 20:33:57.708828 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.708789 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d"} err="failed to get container status \"f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d\": rpc error: code = NotFound desc = could not find container \"f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d\": container with ID starting with f3a4cb4f74428ddfa5dbd0118e0ec94e559d77be49e3bb0e0d7674716c47736d not found: ID does not exist" Jul 15 20:33:57.708828 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.708806 2557 scope.go:117] "RemoveContainer" containerID="25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6" Jul 15 20:33:57.709050 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:33:57.709032 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6\": container with ID starting with 25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6 not found: ID does not exist" containerID="25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6" Jul 15 20:33:57.709106 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.709060 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6"} err="failed to get container status \"25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6\": rpc error: code = NotFound desc = could not find container \"25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6\": container with ID starting with 25cf883c6404a3c159a698627cd8c5744f5a5b14c97b24016a591b8f905729a6 not found: ID does not exist" Jul 15 20:33:57.715991 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.715968 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75"] Jul 15 20:33:57.721252 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:57.721219 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-514f9c-openshift-default-7788f5779f-xrs75"] Jul 15 20:33:58.785894 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:33:58.785856 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" path="/var/lib/kubelet/pods/5f58bdb6-1037-42a7-88bd-8251e697b24e/volumes" Jul 15 20:34:04.003908 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:04.003873 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:08.454591 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:08.454555 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:14.885151 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:14.885104 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:14.959780 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:14.959728 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:19.147816 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.147774 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj"] Jul 15 20:34:19.148425 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.148072 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" containerID="cri-o://92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9" gracePeriod=30 Jul 15 20:34:19.155166 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.155139 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:19.248642 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.248602 2557 patch_prober.go:28] interesting pod/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:34:19.248870 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.248685 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:34:19.378313 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.378282 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:19.714785 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:19.714755 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:20.041651 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.041566 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq"] Jul 15 20:34:20.042132 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042118 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" Jul 15 20:34:20.042174 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042135 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" Jul 15 20:34:20.042174 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042155 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" Jul 15 20:34:20.042174 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042165 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" Jul 15 20:34:20.042316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042264 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" Jul 15 20:34:20.042316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042273 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f58bdb6-1037-42a7-88bd-8251e697b24e" containerName="istio-proxy" Jul 15 20:34:20.042316 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.042281 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c40f77c-f4cf-45f9-b107-f6ed716e5441" containerName="istio-proxy" Jul 15 20:34:20.045557 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.045537 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.048218 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.048181 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:20.049022 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.049002 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-c4ecb2-openshift-default-dockercfg-m28xc\"" Jul 15 20:34:20.061422 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.061396 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq"] Jul 15 20:34:20.177649 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177609 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-credential-socket\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177670 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-envoy\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177773 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-data\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177839 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-podinfo\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177886 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-certs\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177913 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-token\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.177939 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmmfs\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-kube-api-access-hmmfs\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.178024 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b830a9-ac6e-4559-ba6a-42e200ea1438-istiod-ca-cert\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.178157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.178059 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-socket\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.278941 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.278903 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-credential-socket\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.278966 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-envoy\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279003 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-data\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279038 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-podinfo\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279082 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-certs\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279107 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-token\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279129 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279130 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmmfs\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-kube-api-access-hmmfs\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279484 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279191 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b830a9-ac6e-4559-ba6a-42e200ea1438-istiod-ca-cert\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279484 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279227 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-socket\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279600 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279493 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-credential-socket\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279600 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279527 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-certs\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279771 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279749 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-data\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.279879 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.279858 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-socket\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.280142 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.280123 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b830a9-ac6e-4559-ba6a-42e200ea1438-istiod-ca-cert\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.282102 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.282079 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-envoy\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.282318 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.282296 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-podinfo\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.289497 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.289475 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-token\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.289608 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.289569 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmmfs\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-kube-api-access-hmmfs\") pod \"e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.304915 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.304844 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:20.357554 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.357512 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:20.513009 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.512977 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq"] Jul 15 20:34:20.514628 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:34:20.514600 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod14b830a9_ac6e_4559_ba6a_42e200ea1438.slice/crio-d87c0a27da4080579cd0dd3c1feb7ffa74d14523341f2d3644de04f2bd1f5a79 WatchSource:0}: Error finding container d87c0a27da4080579cd0dd3c1feb7ffa74d14523341f2d3644de04f2bd1f5a79: Status 404 returned error can't find the container with id d87c0a27da4080579cd0dd3c1feb7ffa74d14523341f2d3644de04f2bd1f5a79 Jul 15 20:34:20.516985 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.516944 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:20.517068 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.517045 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:20.517113 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.517095 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:20.819062 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.818964 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" event={"ID":"14b830a9-ac6e-4559-ba6a-42e200ea1438","Type":"ContainerStarted","Data":"e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45"} Jul 15 20:34:20.819062 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.819033 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" event={"ID":"14b830a9-ac6e-4559-ba6a-42e200ea1438","Type":"ContainerStarted","Data":"d87c0a27da4080579cd0dd3c1feb7ffa74d14523341f2d3644de04f2bd1f5a79"} Jul 15 20:34:20.843868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:20.843805 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podStartSLOduration=0.843783764 podStartE2EDuration="843.783764ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:34:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:34:20.843382781 +0000 UTC m=+2408.514169333" watchObservedRunningTime="2026-07-15 20:34:20.843783764 +0000 UTC m=+2408.514570290" Jul 15 20:34:21.358462 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:21.358417 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:21.364779 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:21.364752 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:21.825685 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:21.825655 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:21.826667 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:21.826649 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:22.874277 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:22.874214 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:22.970800 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:22.970763 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:23.052680 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:23.052646 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:23.154669 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:23.154567 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:24.004306 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.004272 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:24.101049 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.101017 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:24.409552 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.409527 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:34:24.525019 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.524920 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-credential-socket\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525019 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.524976 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9665df5c-7a84-4f82-99f1-44367313a229-istio-podinfo\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525019 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525014 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-certs\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525346 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525036 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-data\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525346 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525062 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4d4d\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-kube-api-access-n4d4d\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525346 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525167 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-istio-token\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525346 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525211 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9665df5c-7a84-4f82-99f1-44367313a229-istiod-ca-cert\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525346 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525280 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-envoy\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525356 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-socket\") pod \"9665df5c-7a84-4f82-99f1-44367313a229\" (UID: \"9665df5c-7a84-4f82-99f1-44367313a229\") " Jul 15 20:34:24.525638 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525362 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:34:24.525750 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525632 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9665df5c-7a84-4f82-99f1-44367313a229-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:34:24.525826 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525805 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:34:24.525893 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525850 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:34:24.525893 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525862 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.525893 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.525881 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9665df5c-7a84-4f82-99f1-44367313a229-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.526537 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.526498 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-data" (OuterVolumeSpecName: "istio-data") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:34:24.527830 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.527715 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9665df5c-7a84-4f82-99f1-44367313a229-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:34:24.527925 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.527857 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:34:24.527992 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.527980 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-kube-api-access-n4d4d" (OuterVolumeSpecName: "kube-api-access-n4d4d") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "kube-api-access-n4d4d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:34:24.528506 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.528485 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-istio-token" (OuterVolumeSpecName: "istio-token") pod "9665df5c-7a84-4f82-99f1-44367313a229" (UID: "9665df5c-7a84-4f82-99f1-44367313a229"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:34:24.626766 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626726 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.626766 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626758 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.626766 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626768 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9665df5c-7a84-4f82-99f1-44367313a229-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.626766 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626777 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.627034 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626786 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4d4d\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-kube-api-access-n4d4d\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.627034 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626796 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9665df5c-7a84-4f82-99f1-44367313a229-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.627034 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.626804 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9665df5c-7a84-4f82-99f1-44367313a229-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:34:24.844515 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.844473 2557 generic.go:356] "Generic (PLEG): container finished" podID="9665df5c-7a84-4f82-99f1-44367313a229" containerID="92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9" exitCode=0 Jul 15 20:34:24.844692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.844524 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" Jul 15 20:34:24.844692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.844562 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" event={"ID":"9665df5c-7a84-4f82-99f1-44367313a229","Type":"ContainerDied","Data":"92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9"} Jul 15 20:34:24.844692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.844620 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj" event={"ID":"9665df5c-7a84-4f82-99f1-44367313a229","Type":"ContainerDied","Data":"2772116bf8e5f0bfb71a9460c272ec4556186eebe59f156dc4c1674b5a82f894"} Jul 15 20:34:24.844692 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.844649 2557 scope.go:117] "RemoveContainer" containerID="92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9" Jul 15 20:34:24.854003 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.853976 2557 scope.go:117] "RemoveContainer" containerID="c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718" Jul 15 20:34:24.862451 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.862407 2557 scope.go:117] "RemoveContainer" containerID="92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9" Jul 15 20:34:24.862705 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:34:24.862687 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9\": container with ID starting with 92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9 not found: ID does not exist" containerID="92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9" Jul 15 20:34:24.862748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.862714 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9"} err="failed to get container status \"92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9\": rpc error: code = NotFound desc = could not find container \"92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9\": container with ID starting with 92d957cdede4a8ed0d8550bfff7edfe63c46ffde65435d617ea8571096cfc5d9 not found: ID does not exist" Jul 15 20:34:24.862748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.862733 2557 scope.go:117] "RemoveContainer" containerID="c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718" Jul 15 20:34:24.862991 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:34:24.862974 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718\": container with ID starting with c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718 not found: ID does not exist" containerID="c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718" Jul 15 20:34:24.863040 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.862997 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718"} err="failed to get container status \"c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718\": rpc error: code = NotFound desc = could not find container \"c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718\": container with ID starting with c29e1848cf3f5a9a04542f12a4a65716efceec9d44fb720c9f075dc426da5718 not found: ID does not exist" Jul 15 20:34:24.873242 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.873207 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj"] Jul 15 20:34:24.878477 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:24.878449 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-76c4bb-openshift-default-7fb7bc9d5-rpspj"] Jul 15 20:34:26.786126 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:26.786079 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9665df5c-7a84-4f82-99f1-44367313a229" path="/var/lib/kubelet/pods/9665df5c-7a84-4f82-99f1-44367313a229/volumes" Jul 15 20:34:27.335919 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.335885 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:27.678758 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.678727 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:27.975009 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.974922 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t"] Jul 15 20:34:27.975467 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.975384 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" Jul 15 20:34:27.975467 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.975397 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" Jul 15 20:34:27.975594 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.975502 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" Jul 15 20:34:27.980778 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.980747 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:27.983538 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.983518 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-8e78e5-openshift-default-dockercfg-jk7bp\"" Jul 15 20:34:27.995363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:27.995330 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t"] Jul 15 20:34:28.003469 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.003439 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:28.150588 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.150552 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:28.167940 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.167908 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tsxj\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-kube-api-access-9tsxj\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.167969 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-certs\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.167991 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-token\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.168017 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-data\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.168099 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-socket\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168288 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.168142 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-envoy\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168288 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.168192 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-credential-socket\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168288 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.168267 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-podinfo\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.168288 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.168286 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istiod-ca-cert\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269581 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269485 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-credential-socket\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269581 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269524 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-podinfo\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269581 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269545 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istiod-ca-cert\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269832 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269726 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tsxj\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-kube-api-access-9tsxj\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269828 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-certs\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269889 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269862 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-token\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269986 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269913 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-data\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.269986 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269929 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-credential-socket\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.270084 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.269995 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-socket\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.270084 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.270033 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-envoy\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.270294 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.270109 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-certs\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.270294 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.270210 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-data\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.270414 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.270311 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istiod-ca-cert\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.270468 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.270429 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-socket\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.272267 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.272228 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-envoy\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.272375 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.272361 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-podinfo\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.281113 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.281087 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-token\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.281302 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.281212 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tsxj\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-kube-api-access-9tsxj\") pod \"e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.292040 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.292014 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:28.448102 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.448072 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t"] Jul 15 20:34:28.449666 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:34:28.449639 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a74bf93_1d5f_4840_bc74_564c63ef8e15.slice/crio-edb28228f0969b34ca65b64218bdac18e3f3c3dd20ca540a709ee73187d8d391 WatchSource:0}: Error finding container edb28228f0969b34ca65b64218bdac18e3f3c3dd20ca540a709ee73187d8d391: Status 404 returned error can't find the container with id edb28228f0969b34ca65b64218bdac18e3f3c3dd20ca540a709ee73187d8d391 Jul 15 20:34:28.451957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.451918 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:28.452066 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.451989 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:28.452066 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.452021 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:28.868549 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.868453 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" event={"ID":"4a74bf93-1d5f-4840-bc74-564c63ef8e15","Type":"ContainerStarted","Data":"e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9"} Jul 15 20:34:28.868549 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.868504 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" event={"ID":"4a74bf93-1d5f-4840-bc74-564c63ef8e15","Type":"ContainerStarted","Data":"edb28228f0969b34ca65b64218bdac18e3f3c3dd20ca540a709ee73187d8d391"} Jul 15 20:34:28.892996 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:28.892942 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podStartSLOduration=1.8929255459999998 podStartE2EDuration="1.892925546s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 20:34:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:34:28.891138537 +0000 UTC m=+2416.561925076" watchObservedRunningTime="2026-07-15 20:34:28.892925546 +0000 UTC m=+2416.563712073" Jul 15 20:34:29.292826 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:29.292796 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:29.298731 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:29.298703 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:29.874562 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:29.874532 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:29.875521 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:29.875500 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:30.823004 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:30.822916 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:30.925013 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:30.924948 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:31.007743 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:31.007707 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:31.162188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:31.162094 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:31.528661 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:31.528626 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:31.634320 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:31.634212 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:35.289682 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:35.289648 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:36.873549 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.871895 2557 patch_prober.go:28] interesting pod/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:45630->10.132.0.59:15021: read: connection reset by peer" start-of-body= Jul 15 20:34:36.873549 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.871976 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": read tcp 10.132.0.2:45630->10.132.0.59:15021: read: connection reset by peer" Jul 15 20:34:36.922265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.917704 2557 generic.go:356] "Generic (PLEG): container finished" podID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerID="e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45" exitCode=0 Jul 15 20:34:36.922265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.917886 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" event={"ID":"14b830a9-ac6e-4559-ba6a-42e200ea1438","Type":"ContainerDied","Data":"e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45"} Jul 15 20:34:36.922265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.918472 2557 scope.go:117] "RemoveContainer" containerID="e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45" Jul 15 20:34:36.922265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.919788 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:36.922265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.919881 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:36.922265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:36.919929 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:37.934064 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:37.934017 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" event={"ID":"14b830a9-ac6e-4559-ba6a-42e200ea1438","Type":"ContainerStarted","Data":"a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a"} Jul 15 20:34:38.358886 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:38.358775 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:38.359080 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:38.358914 2557 patch_prober.go:28] interesting pod/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 20:34:38.359080 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:38.358973 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 20:34:39.358708 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:39.358606 2557 patch_prober.go:28] interesting pod/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 20:34:39.358708 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:39.358691 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 20:34:40.358748 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:40.358711 2557 patch_prober.go:28] interesting pod/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 20:34:40.359190 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:40.358782 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 20:34:41.358862 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:41.358827 2557 patch_prober.go:28] interesting pod/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" start-of-body= Jul 15 20:34:41.359269 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:41.358888 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.59:15021/healthz/ready\": dial tcp 10.132.0.59:15021: connect: connection refused" Jul 15 20:34:42.361894 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.361860 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:42.362311 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.362273 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:42.363183 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.363161 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:34:42.968815 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.968779 2557 generic.go:356] "Generic (PLEG): container finished" podID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerID="e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9" exitCode=0 Jul 15 20:34:42.969028 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.968856 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" event={"ID":"4a74bf93-1d5f-4840-bc74-564c63ef8e15","Type":"ContainerDied","Data":"e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9"} Jul 15 20:34:42.969473 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.969453 2557 scope.go:117] "RemoveContainer" containerID="e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9" Jul 15 20:34:42.970537 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.970487 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:42.970608 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.970590 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:42.970664 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:42.970643 2557 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 15 20:34:43.976765 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:43.976717 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" event={"ID":"4a74bf93-1d5f-4840-bc74-564c63ef8e15","Type":"ContainerStarted","Data":"dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561"} Jul 15 20:34:44.292722 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:44.292634 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:44.293982 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:44.293953 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 20:34:44.294140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:44.294025 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 20:34:44.855967 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:44.855932 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:44.874729 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:44.874695 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:44.957186 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:44.957153 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:45.293218 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:45.293183 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 20:34:45.293646 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:45.293280 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 20:34:46.293098 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:46.293065 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 20:34:46.293325 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:46.293132 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 20:34:47.293012 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:47.292965 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body= Jul 15 20:34:47.293229 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:47.293025 2557 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" Jul 15 20:34:48.297142 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:48.297108 2557 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:48.298089 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:48.298070 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:34:52.349934 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:52.349896 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:34:52.450193 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:34:52.450159 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:35:01.475716 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:35:01.475676 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:35:35.778784 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:35:35.778746 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:36:05.907543 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:36:05.907452 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:37:41.315883 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:37:41.315800 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:37:42.202548 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:37:42.202516 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:37:51.633253 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:37:51.633120 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:38:09.576362 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:38:09.576323 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:38:14.314684 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:38:14.314650 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:38:16.544095 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:38:16.544062 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:38:52.545625 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:38:52.545582 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:38:52.642260 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:38:52.642198 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:01.732896 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:01.732845 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:03.068454 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:03.068394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:03.201605 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:03.201557 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:21.901216 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:21.901170 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:45.188341 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:45.188291 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:45.785610 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:45.785562 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:45.864070 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:45.864036 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:52.188085 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:52.188056 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:53.293474 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:53.293435 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:39:53.401138 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:39:53.401103 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:40:33.905225 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:40:33.905132 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:40:34.677274 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:40:34.677219 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:40:37.501642 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:40:37.501611 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:40:37.577766 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:40:37.577731 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:40:54.869428 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:40:54.869394 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:40:54.969988 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:40:54.969956 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:04.684062 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:04.684029 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:09.287513 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:09.287472 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:15.355822 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:15.355781 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:15.455931 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:15.455893 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:20.087149 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:20.087114 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq"] Jul 15 20:41:20.087633 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:20.087355 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" containerID="cri-o://a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a" gracePeriod=30 Jul 15 20:41:20.108144 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:20.108109 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:20.347520 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:20.347425 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:21.164554 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:21.164521 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:21.262116 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:21.262081 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:25.348817 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.348763 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:41:25.444787 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.444760 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-data\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.444787 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.444789 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-certs\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.444957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.444858 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-socket\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.444957 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.444927 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-credential-socket\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.445040 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.444973 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-podinfo\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.445040 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445030 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-token\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.445140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445060 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-envoy\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.445140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445083 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmmfs\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-kube-api-access-hmmfs\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.445140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445122 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b830a9-ac6e-4559-ba6a-42e200ea1438-istiod-ca-cert\") pod \"14b830a9-ac6e-4559-ba6a-42e200ea1438\" (UID: \"14b830a9-ac6e-4559-ba6a-42e200ea1438\") " Jul 15 20:41:25.445140 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445117 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:25.445377 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445197 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:25.445377 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445220 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:25.445579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445517 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.445579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445544 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.445579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445551 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/14b830a9-ac6e-4559-ba6a-42e200ea1438-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:41:25.445579 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445558 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.445802 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.445612 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-data" (OuterVolumeSpecName: "istio-data") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:25.447417 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.447396 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:41:25.447558 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.447540 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-token" (OuterVolumeSpecName: "istio-token") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:41:25.447770 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.447751 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:25.447861 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.447801 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-kube-api-access-hmmfs" (OuterVolumeSpecName: "kube-api-access-hmmfs") pod "14b830a9-ac6e-4559-ba6a-42e200ea1438" (UID: "14b830a9-ac6e-4559-ba6a-42e200ea1438"). InnerVolumeSpecName "kube-api-access-hmmfs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:41:25.546178 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.546151 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.546178 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.546174 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.546359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.546184 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.546359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.546198 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmmfs\" (UniqueName: \"kubernetes.io/projected/14b830a9-ac6e-4559-ba6a-42e200ea1438-kube-api-access-hmmfs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.546359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.546212 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/14b830a9-ac6e-4559-ba6a-42e200ea1438-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:25.546359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:25.546221 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/14b830a9-ac6e-4559-ba6a-42e200ea1438-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:26.208815 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.208776 2557 generic.go:356] "Generic (PLEG): container finished" podID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerID="a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a" exitCode=0 Jul 15 20:41:26.209017 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.208828 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" Jul 15 20:41:26.209017 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.208852 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" event={"ID":"14b830a9-ac6e-4559-ba6a-42e200ea1438","Type":"ContainerDied","Data":"a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a"} Jul 15 20:41:26.209017 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.208907 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq" event={"ID":"14b830a9-ac6e-4559-ba6a-42e200ea1438","Type":"ContainerDied","Data":"d87c0a27da4080579cd0dd3c1feb7ffa74d14523341f2d3644de04f2bd1f5a79"} Jul 15 20:41:26.209017 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.208931 2557 scope.go:117] "RemoveContainer" containerID="a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a" Jul 15 20:41:26.219289 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.219267 2557 scope.go:117] "RemoveContainer" containerID="e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45" Jul 15 20:41:26.227682 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.227667 2557 scope.go:117] "RemoveContainer" containerID="a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a" Jul 15 20:41:26.227917 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:41:26.227901 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a\": container with ID starting with a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a not found: ID does not exist" containerID="a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a" Jul 15 20:41:26.227969 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.227928 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a"} err="failed to get container status \"a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a\": rpc error: code = NotFound desc = could not find container \"a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a\": container with ID starting with a5c272d4756e801d476bbc0329f2e482c242ef0c04b772c2c9a5850fe7ead23a not found: ID does not exist" Jul 15 20:41:26.227969 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.227944 2557 scope.go:117] "RemoveContainer" containerID="e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45" Jul 15 20:41:26.228152 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:41:26.228139 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45\": container with ID starting with e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45 not found: ID does not exist" containerID="e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45" Jul 15 20:41:26.228193 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.228154 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45"} err="failed to get container status \"e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45\": rpc error: code = NotFound desc = could not find container \"e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45\": container with ID starting with e239f9b26fb0384dbf0033284f170524a578d347bd818b11321908fee31a3b45 not found: ID does not exist" Jul 15 20:41:26.247643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.247620 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq"] Jul 15 20:41:26.249738 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.249714 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-c4ecb2-openshift-default-76869b75ff-zrvxq"] Jul 15 20:41:26.787069 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:26.787035 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" path="/var/lib/kubelet/pods/14b830a9-ac6e-4559-ba6a-42e200ea1438/volumes" Jul 15 20:41:31.075820 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:31.075769 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:35.605667 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:35.605625 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:42.244206 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:42.244168 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:42.329768 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:42.329731 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:46.298522 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:46.298480 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t"] Jul 15 20:41:46.298944 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:46.298715 2557 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" containerID="cri-o://dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561" gracePeriod=30 Jul 15 20:41:46.326441 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:46.326415 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:46.538606 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:46.538560 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:47.427172 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:47.427134 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:41:48.300364 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:48.300328 2557 patch_prober.go:28] interesting pod/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 15 20:41:48.300527 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:48.300392 2557 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 20:41:51.560988 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.560964 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:41:51.684539 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684516 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tsxj\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-kube-api-access-9tsxj\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684545 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-certs\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684574 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-envoy\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684595 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-token\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684616 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-credential-socket\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684674 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684649 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-socket\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684920 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684725 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istiod-ca-cert\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684920 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684802 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-podinfo\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684920 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684833 2557 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-data\") pod \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\" (UID: \"4a74bf93-1d5f-4840-bc74-564c63ef8e15\") " Jul 15 20:41:51.684920 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684839 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:51.685123 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.684981 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:51.685179 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.685126 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:51.685256 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.685126 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 20:41:51.685256 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.685214 2557 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-certs\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.685357 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.685273 2557 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-workload-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.686030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.686005 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-data" (OuterVolumeSpecName: "istio-data") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:51.686977 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.686945 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-token" (OuterVolumeSpecName: "istio-token") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:41:51.687088 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.686973 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 20:41:51.687357 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.687338 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 15 20:41:51.687471 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.687451 2557 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-kube-api-access-9tsxj" (OuterVolumeSpecName: "kube-api-access-9tsxj") pod "4a74bf93-1d5f-4840-bc74-564c63ef8e15" (UID: "4a74bf93-1d5f-4840-bc74-564c63ef8e15"). InnerVolumeSpecName "kube-api-access-9tsxj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 20:41:51.786309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786284 2557 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-envoy\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.786309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786304 2557 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-token\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.786309 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786313 2557 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-credential-socket\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.786472 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786321 2557 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istiod-ca-cert\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.786472 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786331 2557 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-podinfo\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.786472 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786338 2557 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4a74bf93-1d5f-4840-bc74-564c63ef8e15-istio-data\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:51.786472 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:51.786348 2557 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tsxj\" (UniqueName: \"kubernetes.io/projected/4a74bf93-1d5f-4840-bc74-564c63ef8e15-kube-api-access-9tsxj\") on node \"ip-10-0-132-12.ec2.internal\" DevicePath \"\"" Jul 15 20:41:52.364282 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.364250 2557 generic.go:356] "Generic (PLEG): container finished" podID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerID="dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561" exitCode=0 Jul 15 20:41:52.364470 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.364293 2557 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" Jul 15 20:41:52.364470 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.364325 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" event={"ID":"4a74bf93-1d5f-4840-bc74-564c63ef8e15","Type":"ContainerDied","Data":"dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561"} Jul 15 20:41:52.364470 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.364377 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t" event={"ID":"4a74bf93-1d5f-4840-bc74-564c63ef8e15","Type":"ContainerDied","Data":"edb28228f0969b34ca65b64218bdac18e3f3c3dd20ca540a709ee73187d8d391"} Jul 15 20:41:52.364470 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.364399 2557 scope.go:117] "RemoveContainer" containerID="dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561" Jul 15 20:41:52.374393 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.374375 2557 scope.go:117] "RemoveContainer" containerID="e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9" Jul 15 20:41:52.382490 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.382475 2557 scope.go:117] "RemoveContainer" containerID="dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561" Jul 15 20:41:52.382756 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:41:52.382739 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561\": container with ID starting with dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561 not found: ID does not exist" containerID="dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561" Jul 15 20:41:52.382805 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.382764 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561"} err="failed to get container status \"dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561\": rpc error: code = NotFound desc = could not find container \"dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561\": container with ID starting with dc46e2f385fb707be4ba84c26108425b09c85ee27bc3bca6585da47b67462561 not found: ID does not exist" Jul 15 20:41:52.382805 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.382779 2557 scope.go:117] "RemoveContainer" containerID="e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9" Jul 15 20:41:52.383014 ip-10-0-132-12 kubenswrapper[2557]: E0715 20:41:52.382993 2557 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9\": container with ID starting with e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9 not found: ID does not exist" containerID="e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9" Jul 15 20:41:52.383071 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.383024 2557 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9"} err="failed to get container status \"e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9\": rpc error: code = NotFound desc = could not find container \"e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9\": container with ID starting with e9854e640c37d3ae35387f95d49dff29c3b4ad9d2414a60c739962ff38fbc3a9 not found: ID does not exist" Jul 15 20:41:52.386726 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.386707 2557 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t"] Jul 15 20:41:52.390543 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.390527 2557 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-8e78e5-openshift-default-5677756787-ht95t"] Jul 15 20:41:52.515157 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.515129 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-psv5z_5ddeb833-fb22-40bc-a670-8ff1c80c8197/manager/0.log" Jul 15 20:41:52.645112 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.645032 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bb97q_f0632d30-c3aa-4105-8f6b-4b7a7e4a5796/manager/0.log" Jul 15 20:41:52.786195 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:52.786164 2557 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" path="/var/lib/kubelet/pods/4a74bf93-1d5f-4840-bc74-564c63ef8e15/volumes" Jul 15 20:41:53.140170 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:53.140145 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-8cwmc_27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae/copy-manifests/0.log" Jul 15 20:41:53.159576 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:53.159543 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-8cwmc_27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae/manager/0.log" Jul 15 20:41:53.859487 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:53.859453 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735775-hz8gt_07bb68c9-fde9-44f1-bc56-684e8031e6cf/cleanup/0.log" Jul 15 20:41:53.973820 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:53.973790 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29735790-6lslf_8707a2ed-31bf-443b-a585-8b760e22f1e4/cleanup/0.log" Jul 15 20:41:54.195090 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:54.195059 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-ait-dc00279d-xw6fq_684249a5-1145-4d4a-bf32-eea838a01ac4/revoke-keys/1.log" Jul 15 20:41:54.991193 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:54.991157 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-mt-83c1a2fd-8j9p7_f4e4bfab-2444-4ff8-8236-c5bb0f29534f/revoke-keys/0.log" Jul 15 20:41:56.847837 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:56.847798 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mbxmt_b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f/manager/0.log" Jul 15 20:41:57.196279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:57.196201 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-s7zbn_c84a0ec2-070f-45fc-a1ac-f0c5456ad889/registry-server/0.log" Jul 15 20:41:57.558169 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:57.558080 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v5dss_1a27705c-072f-4fa1-8659-764e557b6759/manager/0.log" Jul 15 20:41:57.911179 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:57.911090 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wh6nx_c1928ce8-9c2f-4207-8581-e0b1fa3c1381/istio-proxy/0.log" Jul 15 20:41:58.389121 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:58.389089 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/1.log" Jul 15 20:41:58.510809 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:58.510782 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-rqlj9_8dcbf538-3f10-4aca-a98c-429c13a73ca2/payload-pre-processing/0.log" Jul 15 20:41:58.633294 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:58.633251 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-xln9v_73b159e8-12be-4899-8aa8-55fb08f88476/payload-processing/0.log" Jul 15 20:41:59.298363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:41:59.298335 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-nb4cv_978ecb18-4960-428c-a75a-4df003f0014e/main/0.log" Jul 15 20:42:05.204271 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:05.204154 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:42:05.294258 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:05.294203 2557 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" limit=1024 Jul 15 20:42:08.470159 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:08.470128 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tr84l_4e54a4d4-ffa7-4e74-8cad-960dceb020ab/global-pull-secret-syncer/0.log" Jul 15 20:42:08.532081 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:08.532044 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-72vbr_c8296fea-0715-4071-8570-f4301f99baf7/konnectivity-agent/0.log" Jul 15 20:42:08.637702 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:08.637677 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-12.ec2.internal_a843563177c22b96d13b44f8b4e55822/haproxy/0.log" Jul 15 20:42:13.493127 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:13.493092 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-mbxmt_b8ccfe76-91f8-4ab3-96e6-5c1cdac2359f/manager/0.log" Jul 15 20:42:13.627166 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:13.626932 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-s7zbn_c84a0ec2-070f-45fc-a1ac-f0c5456ad889/registry-server/0.log" Jul 15 20:42:13.877165 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:13.877080 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-v5dss_1a27705c-072f-4fa1-8659-764e557b6759/manager/0.log" Jul 15 20:42:15.792635 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.792562 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/alertmanager/0.log" Jul 15 20:42:15.822950 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.822911 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/config-reloader/0.log" Jul 15 20:42:15.847867 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.847846 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/kube-rbac-proxy-web/0.log" Jul 15 20:42:15.874061 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.874039 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/kube-rbac-proxy/0.log" Jul 15 20:42:15.898381 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.898356 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/kube-rbac-proxy-metric/0.log" Jul 15 20:42:15.925677 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.925650 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/prom-label-proxy/0.log" Jul 15 20:42:15.956396 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:15.956375 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e0a8ab20-b46f-4fb5-9fee-f932db077071/init-config-reloader/0.log" Jul 15 20:42:16.009791 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.009766 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-gqj67_6d0326db-fe6e-4e6f-b8bf-fb8303d78358/cluster-monitoring-operator/0.log" Jul 15 20:42:16.040161 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.040142 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-drrqp_293af30d-2d20-4958-accf-5902f7005566/kube-state-metrics/0.log" Jul 15 20:42:16.062754 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.062685 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-drrqp_293af30d-2d20-4958-accf-5902f7005566/kube-rbac-proxy-main/0.log" Jul 15 20:42:16.085937 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.085913 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-drrqp_293af30d-2d20-4958-accf-5902f7005566/kube-rbac-proxy-self/0.log" Jul 15 20:42:16.115724 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.115704 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-597f6b7d7d-bv76z_0f8f0728-7d76-4422-936e-03b6c92cabbb/metrics-server/0.log" Jul 15 20:42:16.148683 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.148633 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-mbjjl_80159a3f-84dc-4a89-9973-c863520fb525/monitoring-plugin/0.log" Jul 15 20:42:16.400643 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.400562 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjvbj_029d47f2-a823-4943-a139-6c7256afc076/node-exporter/0.log" Jul 15 20:42:16.427279 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.427259 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjvbj_029d47f2-a823-4943-a139-6c7256afc076/kube-rbac-proxy/0.log" Jul 15 20:42:16.449640 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.449623 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qjvbj_029d47f2-a823-4943-a139-6c7256afc076/init-textfile/0.log" Jul 15 20:42:16.479462 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.479441 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-47w89_ee492500-72c2-497a-8d75-856b40542110/kube-rbac-proxy-main/0.log" Jul 15 20:42:16.501116 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.501097 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-47w89_ee492500-72c2-497a-8d75-856b40542110/kube-rbac-proxy-self/0.log" Jul 15 20:42:16.523690 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.523673 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-47w89_ee492500-72c2-497a-8d75-856b40542110/openshift-state-metrics/0.log" Jul 15 20:42:16.563110 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.563089 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/prometheus/0.log" Jul 15 20:42:16.583785 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.583765 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/config-reloader/0.log" Jul 15 20:42:16.604593 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.604572 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/thanos-sidecar/0.log" Jul 15 20:42:16.627197 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.627177 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/kube-rbac-proxy-web/0.log" Jul 15 20:42:16.654126 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.654051 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/kube-rbac-proxy/0.log" Jul 15 20:42:16.679623 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.679603 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/kube-rbac-proxy-thanos/0.log" Jul 15 20:42:16.707713 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.707696 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1668ee16-4826-4fa6-a35a-2641ebc17919/init-config-reloader/0.log" Jul 15 20:42:16.744904 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.744884 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-wcf9q_7a973d49-e6c6-4b6b-a698-6a3011c7c5e5/prometheus-operator/0.log" Jul 15 20:42:16.768958 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.768936 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-wcf9q_7a973d49-e6c6-4b6b-a698-6a3011c7c5e5/kube-rbac-proxy/0.log" Jul 15 20:42:16.801795 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.801768 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-t29t9_93a015ac-a73a-4564-b664-f44768cc7667/prometheus-operator-admission-webhook/0.log" Jul 15 20:42:16.836685 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.836668 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-665cff56f5-p9tl4_68085e92-7344-4419-b0f9-99ca5ebe54ee/telemeter-client/0.log" Jul 15 20:42:16.859164 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.859139 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-665cff56f5-p9tl4_68085e92-7344-4419-b0f9-99ca5ebe54ee/reload/0.log" Jul 15 20:42:16.881375 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.881352 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-665cff56f5-p9tl4_68085e92-7344-4419-b0f9-99ca5ebe54ee/kube-rbac-proxy/0.log" Jul 15 20:42:16.912191 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.912122 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f7cf95d8f-62pxv_86d1f514-a169-40f4-9745-243de995e52a/thanos-query/0.log" Jul 15 20:42:16.937420 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.937399 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f7cf95d8f-62pxv_86d1f514-a169-40f4-9745-243de995e52a/kube-rbac-proxy-web/0.log" Jul 15 20:42:16.959395 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.959368 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f7cf95d8f-62pxv_86d1f514-a169-40f4-9745-243de995e52a/kube-rbac-proxy/0.log" Jul 15 20:42:16.980359 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:16.980338 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f7cf95d8f-62pxv_86d1f514-a169-40f4-9745-243de995e52a/prom-label-proxy/0.log" Jul 15 20:42:17.009396 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:17.009375 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f7cf95d8f-62pxv_86d1f514-a169-40f4-9745-243de995e52a/kube-rbac-proxy-rules/0.log" Jul 15 20:42:17.064382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:17.064365 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6f7cf95d8f-62pxv_86d1f514-a169-40f4-9745-243de995e52a/kube-rbac-proxy-metrics/0.log" Jul 15 20:42:18.226512 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.226383 2557 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42"] Jul 15 20:42:18.226980 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.226962 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" Jul 15 20:42:18.227030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.226983 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" Jul 15 20:42:18.227030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.226990 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" Jul 15 20:42:18.227030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.226996 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" Jul 15 20:42:18.227030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227004 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" Jul 15 20:42:18.227030 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227009 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" Jul 15 20:42:18.227188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227045 2557 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" Jul 15 20:42:18.227188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227051 2557 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" Jul 15 20:42:18.227188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227129 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a74bf93-1d5f-4840-bc74-564c63ef8e15" containerName="istio-proxy" Jul 15 20:42:18.227188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227142 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="9665df5c-7a84-4f82-99f1-44367313a229" containerName="istio-proxy" Jul 15 20:42:18.227188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227149 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" Jul 15 20:42:18.227188 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.227154 2557 memory_manager.go:368] "RemoveStaleState removing state" podUID="14b830a9-ac6e-4559-ba6a-42e200ea1438" containerName="istio-proxy" Jul 15 20:42:18.232265 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.232225 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.234764 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.234738 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mncbm\"/\"openshift-service-ca.crt\"" Jul 15 20:42:18.234907 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.234774 2557 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-mncbm\"/\"default-dockercfg-f7qnm\"" Jul 15 20:42:18.235725 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.235705 2557 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-mncbm\"/\"kube-root-ca.crt\"" Jul 15 20:42:18.241916 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.241889 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42"] Jul 15 20:42:18.244072 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.244048 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-podres\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.244183 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.244091 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-sys\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.244183 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.244161 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trxvv\" (UniqueName: \"kubernetes.io/projected/f025dc12-a2ce-477c-86ea-8a5ddd3af220-kube-api-access-trxvv\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.244329 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.244196 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-lib-modules\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.244329 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.244270 2557 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-proc\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345169 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345136 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-proc\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345204 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-podres\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345261 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-sys\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345283 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-proc\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345311 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-trxvv\" (UniqueName: \"kubernetes.io/projected/f025dc12-a2ce-477c-86ea-8a5ddd3af220-kube-api-access-trxvv\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345335 2557 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-lib-modules\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345336 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-sys\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345382 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345376 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-podres\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.345669 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.345496 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f025dc12-a2ce-477c-86ea-8a5ddd3af220-lib-modules\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.354113 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.354085 2557 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-trxvv\" (UniqueName: \"kubernetes.io/projected/f025dc12-a2ce-477c-86ea-8a5ddd3af220-kube-api-access-trxvv\") pod \"perf-node-gather-daemonset-q8d42\" (UID: \"f025dc12-a2ce-477c-86ea-8a5ddd3af220\") " pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.543796 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.543693 2557 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:18.680977 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:18.680951 2557 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42"] Jul 15 20:42:18.683515 ip-10-0-132-12 kubenswrapper[2557]: W0715 20:42:18.683487 2557 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf025dc12_a2ce_477c_86ea_8a5ddd3af220.slice/crio-4f3c4bc3f4c638fbc38062af8a4ae4b60143f24f04fd3f5875fe85d6d4b376ec WatchSource:0}: Error finding container 4f3c4bc3f4c638fbc38062af8a4ae4b60143f24f04fd3f5875fe85d6d4b376ec: Status 404 returned error can't find the container with id 4f3c4bc3f4c638fbc38062af8a4ae4b60143f24f04fd3f5875fe85d6d4b376ec Jul 15 20:42:19.456390 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:19.456364 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6c5bd6f84f-l9jqn_8e3381e0-5e8c-43dd-bd86-eb3f8e200468/console/0.log" Jul 15 20:42:19.515378 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:19.515351 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" event={"ID":"f025dc12-a2ce-477c-86ea-8a5ddd3af220","Type":"ContainerStarted","Data":"9533ac41fda2e334e719198710e85331f87395707b852ba3f17536103083931c"} Jul 15 20:42:19.515518 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:19.515400 2557 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" event={"ID":"f025dc12-a2ce-477c-86ea-8a5ddd3af220","Type":"ContainerStarted","Data":"4f3c4bc3f4c638fbc38062af8a4ae4b60143f24f04fd3f5875fe85d6d4b376ec"} Jul 15 20:42:19.515518 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:19.515426 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:19.536255 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:19.536184 2557 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" podStartSLOduration=1.536163497 podStartE2EDuration="1.536163497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 20:42:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 20:42:19.534504581 +0000 UTC m=+2887.205291108" watchObservedRunningTime="2026-07-15 20:42:19.536163497 +0000 UTC m=+2887.206950024" Jul 15 20:42:21.270794 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:21.270765 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f7wqb_2602891b-0d4f-4354-964e-c028e469d84b/dns/0.log" Jul 15 20:42:21.297495 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:21.297469 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-f7wqb_2602891b-0d4f-4354-964e-c028e469d84b/kube-rbac-proxy/0.log" Jul 15 20:42:21.381172 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:21.381144 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4gk5v_0303819b-d5e8-4964-94ed-2b90452a5036/dns-node-resolver/0.log" Jul 15 20:42:22.050713 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:22.050684 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-5l4v9_c14a3660-2a88-490e-b481-6a414a642760/node-ca/0.log" Jul 15 20:42:23.160420 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:23.160383 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wh6nx_c1928ce8-9c2f-4207-8581-e0b1fa3c1381/istio-proxy/0.log" Jul 15 20:42:23.391510 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:23.391471 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/0.log" Jul 15 20:42:23.452043 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:23.452006 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/1.log" Jul 15 20:42:23.488861 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:23.488812 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-rqlj9_8dcbf538-3f10-4aca-a98c-429c13a73ca2/payload-pre-processing/0.log" Jul 15 20:42:23.531395 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:23.531366 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-xln9v_73b159e8-12be-4899-8aa8-55fb08f88476/payload-processing/0.log" Jul 15 20:42:24.363895 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:24.363867 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-blcpq_1f70d4bd-946c-415a-b6c7-6546d7760288/serve-healthcheck-canary/0.log" Jul 15 20:42:24.929211 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:24.929179 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5dklm_f9c10eff-c27b-48a5-bb31-ede810a60d26/kube-rbac-proxy/0.log" Jul 15 20:42:24.954610 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:24.954579 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5dklm_f9c10eff-c27b-48a5-bb31-ede810a60d26/exporter/0.log" Jul 15 20:42:24.980018 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:24.979990 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5dklm_f9c10eff-c27b-48a5-bb31-ede810a60d26/extractor/0.log" Jul 15 20:42:25.531621 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:25.531593 2557 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-mncbm/perf-node-gather-daemonset-q8d42" Jul 15 20:42:27.613991 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:27.613963 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-psv5z_5ddeb833-fb22-40bc-a670-8ff1c80c8197/manager/0.log" Jul 15 20:42:27.860974 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:27.860886 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-bb97q_f0632d30-c3aa-4105-8f6b-4b7a7e4a5796/manager/0.log" Jul 15 20:42:28.208467 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.208435 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-8cwmc_27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae/manager/0.log" Jul 15 20:42:28.232739 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.232714 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-57c4dc5856-8cwmc_27a2a7fc-a7ea-4d1d-b92c-b5fa87f2a6ae/copy-manifests/0.log" Jul 15 20:42:28.523328 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.523205 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wh6nx_c1928ce8-9c2f-4207-8581-e0b1fa3c1381/istio-proxy/0.log" Jul 15 20:42:28.721740 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.721709 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/0.log" Jul 15 20:42:28.785483 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.785394 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/1.log" Jul 15 20:42:28.811610 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.811584 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-rqlj9_8dcbf538-3f10-4aca-a98c-429c13a73ca2/payload-pre-processing/0.log" Jul 15 20:42:28.846363 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:28.846329 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-xln9v_73b159e8-12be-4899-8aa8-55fb08f88476/payload-processing/0.log" Jul 15 20:42:29.674461 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:29.674435 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b4664dc78-msgg9_5bed4249-443e-4f02-8ba2-22e57169577e/manager/0.log" Jul 15 20:42:30.171503 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:30.171421 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-7f964fb678wh6nx_c1928ce8-9c2f-4207-8581-e0b1fa3c1381/istio-proxy/0.log" Jul 15 20:42:30.377505 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:30.377473 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/0.log" Jul 15 20:42:30.427580 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:30.427547 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5896d4dd49-c5jhx_c5690746-549e-4381-85cb-a74a922f05e3/istio-proxy/1.log" Jul 15 20:42:30.479373 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:30.479347 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-rqlj9_8dcbf538-3f10-4aca-a98c-429c13a73ca2/payload-pre-processing/0.log" Jul 15 20:42:30.509754 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:30.509727 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-f9b8d9b9b-xln9v_73b159e8-12be-4899-8aa8-55fb08f88476/payload-processing/0.log" Jul 15 20:42:34.899868 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:34.899833 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b4jps_69dbd07b-dbcd-427f-b2e8-ac27e5f67545/migrator/0.log" Jul 15 20:42:34.921041 ip-10-0-132-12 kubenswrapper[2557]: I0715 20:42:34.921015 2557 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-b4jps_69dbd07b-dbcd-427f-b2e8-ac27e5f67545/graceful-termination/0.log"